DEFINITION OF AUGMENTED REALITY
THE PROJECT
GOALS
MAIN RESEARCH TOPICS


Definition of Augmented Reality
Augmented reality (AR) is reality superimposed by virtual reality (VR). The system allows the user to see virtual objects, which are embedded in the physical environment. While "virtual reality" completely immerses the user in an illusionary synthetic world, "augmented reality" just adds synthetic elements to the user’s perception of the real world.


The Project
The project title living-room is ambiguous. It can be understood to be at once a living-room and a room that lives. This research project tries to fuse the two concepts with the aim of creating a dynamically extended room with the application of AR-technology.
There is a specially constructed real space wherein the players can move with a HMD (Head Mounted Display). The special approach of living-room is the immersive application. In comparison to known examples where objects can be moved around, the user him/herself moves in space where the objects are placed or animated. This space can be visually transformed, extended to infinity, populated, etc. The arising possibilities open new ways for applications in the fields of Architecture, Scenography, Tourism and Museology.
There are several artistic scenes described under the menu item “scenarios”. Those examples where elaborated to examine different aspects of the hybrid physical-digital space. Perception and meaning of the room become influenced by movement, contrast, scale, colour, transparency/opacity, etc. User interaction is implemented by using the user position needed for the tracking for triggering scene elements too: the room comes to life.
The room can be dismantled to be exposed at events for art and technology. It will be further developed and will become part of an educational process.
 

Goals
The goal of „living-room2“ is to acquire a Design-Vocabulary for room-oriented AR-scenarios and the technical development of a hybrid tracking system as well as a Designer’s Toolbox which are fundamental for the research of design aspects.
The former research project “living-room1” served as a proof-of-concept and evaluation phase. The needs for the creation of a fundament for AR-Design were proved and the main aspects spotted: Physical/Virtual, Interaction, Dramaturgy. The Design-Vocabulary is built by means of theoretical examination, practical experiments, verification and documentation.
The technical innovation for room-oriented AR is the development of a novel hybrid tracking system (optical object tracking plus ultra sonic and inertial tracking for position and direction) and the Designer’s Toolbox.


Main research topics
EDGES AND LINKS BETWEEN THE REAL AND THE VIRTUAL – CREATING A NEW REALITY

Appearance:
The boundaries between the real and the virtual world shall disappear. To bring them closer together not only the virtual world will be designed as real as possible, but the real world gets treated in real-time as well to make it look more similar to the virtual world.
- Exploration: possibility of real-time treatment of textures (video backdrop) within Virtools (colour, light, contrast, etc.)

The visual mix between the worlds is created by real-time occlusion and shadow calculations, as much as by transparency.
-
Task: the real world setup needs to be measured and occlusion and shadow implemented. Optimisation of the precision of the boundaries.
- Tests: with transparency on different grounds (white, colour, mirror, backlit fabric, etc.)
- Exploration: feather effects at the pixel boarders, other visual tricks to avoid inaccuracy detection

Virtual 3D-Sound is getting mixed with real sounds (television, telephone, noise of element in wind of ventilator, etc.)
- Task: positioning 3D-sound in context with virtual and real audio and visual elements


Interaction between the worlds:
Interaction measured in the real world (tracking, sensing) will have effect in both worlds. It can alter virtual elements, atmospheres and their logic or can influence physical parameters of sound, light, electromagnetism, etc. The logic of the virtual system can regenerate itself and have effect on physical elements.
- Real-real: is understood as the normal daily life experience, these human experiences and interaction possibilities get extended by additional capacities of the prepared room (sensors and tracking devices) to influence on physical parameters of sound, light, electromagnetism, etc. of the same room
-
Real-virtual: by tracking and sensing user behaviour the overlaid virtual elements can be influenced
-
Virtual-real: artificial intelligence or definitions of combinations of elements and states of the system influencing on physical elements)
-
Virtual-virtual: artificial intelligence, relations between the virtual elements create new virtual situations

- Exploration: all 4 combinations should be explored and brought into easily recognisable scenarios


Seamless contextualisation and story telling between the worlds:
Elements from the real and virtual world form a ensemble to tell stories or contribute to a common expression.
-
Exploration: composition of time and space related mixed AR-media


INTERACTION WITHIN AR

Interaction with interfaces (sensors for objects):
Context with sensor objects.

Interaction by position and look-at-direction (tracked visitor):
Context to the field of view.

Switching the scenarios:
Two navigation systems get explored and evaluated: handheld steering device and game-like links from one scenario (world) to the other.
When passing over to a new scenario virtual elements might stay at their place getting in a new context.


STAGING AR

Since the viewer can’t see spatially dispersed audiovisual elements happening in different places at the same time, time-based elements are presented as loops or are returning repeatedly. They can be displayed in time and place according to content and narration.

Spatially positioned sound helps to detect elements that are not found in the field of view.
-
Exploration: construction of time and space related audiovisual elements forming an ensemble
-
Exploration: narrative structures within mixed AR-media
-
Exploration: game structures within mixed AR-media (point system, solving riddles, stepping into other worlds after completion of mission, etc.)