• People
• ITziouvara

Ioanna Tziouvara

MSc student in Computer Science
track: Media and Knowledge Engineering

e-mail: joan.tziouvara@gmail.com

Master’s thesis project: Integration of 3D Tracking systems for Interaction in Spatial Augmented Reality

Summary:
In this project two different 3D tracking systems, namely an infrared optical tracker and an RGB-D camera, are combined in order to support interaction for projected augmented reality applications. This work entails RGB-D camera to real world calibration, projector as a virtual camera to the real world calibration, user input via a tracked pen, a spatial menu as user interface enabling selection and annotation functionalities, as well as a 3D object scanner registering obtained instances from the 3D tracking systems.

A use case of the system:
During the very first stages of prototyping, a designer wants to scan a freshly handmade object and iterate through the design process of that object. Our 3D scanner provides this possibility by getting a number of instances from the RGB-D camera in the form of point clouds, then registering these point clouds with the help of the infrared tracker to produce one total point cloud, which contains every single view angle of the 3D object. After some post-filtering, the point cloud is converted to a polygonal mesh in order to support texturing. This mesh is then projected on the real object with the help of our digital projector. In addition, a tracked spatial menu offers selection and annotation functionality. This means that the designer can select to use a number of different colors for annotating on the object’s surface or on the ground plane of the world, change the color of the object and select materials. The user input is inserted by a tracked pen with the use of retro-reflective stickers. The graphics generated by the projector (mesh, menu, annotations) follow the tracked objects in real time. For a visual hint of the SAR system and the scanner setup see the figure below.

INSYGHT demo/First Pilot test:
During the opening of the INSYGHT lab, we will carry out the very first pilot of the system in order to test the operation of some of its parts, get feedback for further improvements or changes and last but not least take the chance to give a demo of the system. We mainly focus on the system's scanning application and the interaction with the scanned object and/or the world. Due to the fact that the system is in development phase, possible bugs and weird behavior will be noted and fixed before the release of the next version. Some of the tasks that will be tested/demonstrated are:

• a) thePreLoop (setting Local Coordinates in point cloud)
  In this test, the user clicks on two 3d points and the transformation between the RGB-D camera and the cheese platter coordinates is automatically computed (figure below shows the result of this process).

• b) theLoop (Scanning, Registration and Meshing)
  In this test the user rotates the cheese platter, so that the object on it gets scanned and registered. When this process completes, an entire point cloud of the object and a polygonal mesh of that point cloud are given as output (figures below show the results of this process).

• c) thePostLoop (Interaction with model and the world)
  A spatial menu that enables change of color for annotation and change of textures for a given 3d-print model is used as a form of graphical user interface. The user input is accomplished though a tracked pen, which allows interaction with the scanned model, the 3d-print model and the world (figures below show the spatial menu and interaction with the world).