Research topics
SYNCHRONIZATION OF THE REAL AND THE VIRTUAL – CREATING A NEW REALITY

Aesthetics / appearance
‘living-room2’ intends to synchronize the real and the virtual by applying the specific properties of both worlds, exploring visual aesthetics, sound, interaction and narrative. The goal is to design a new user experience, in which the borders between physical and virtual have become unperceivable and invisible.

Visual aesthetics
The aim of ‘living-room2’ is to develop a common visual language in order to connect the different realities. Additionally, real-time treatment of the physical environment is used to synchronize both worlds.
- Exploration: possibility of real-time treatment of textures (video backdrop) within Virtools (color, light, contrast, ...)

- Degeneration of the quality of the 3D-Render (blur, noise, compression artefacts, ...)

Visually the different worlds will be connected by applying real-time occlusion, shadow casting, transparency, light position and light temperature.
- Task: the real world setup needs to be measured and occlusion and shadow implemented. Optimizing of the precision of the boundaries.
- Tests: with transparency and keying on different grounds (white, color, mirror, backlit fabric, etc.)
- Exploration: feather mask for the blending of video background and 3D-foreground, other visual tricks to avoid inaccuracy detection

Sound
Virtual 3D-Sound is mixed with real sounds (noise of element in wind of ventilator, etc.)
Task: positioning 3D-sound in context with virtual and real audio and visual elements
Sound for narrative and guidance.

Interaction between the worlds (see scenarios)
Interaction measured in the real world (tracking, sensing) will have effect in both worlds. It can change virtual elements, atmospheres and their logic can affect physical parameters of sound, light, electromagnetism, etc. The logic of the virtual system can regenerate itself and influence physical elements.
- Real-real: is understood as the normal daily life experience. These human experiences and interaction possibilities will be extended by additional capacities of the prepared room (sensors and tracking devices) to influence on physical parameters of sound, light, electromagnetism, etc. of the same room.
- Real-virtual: by tracking and sensing user behavior, the overlaid virtual elements can be influenced.
- Virtual-real: simulation of artificial intelligence or definitions of combinations of elements and states of the system affecting physical elements like real lamps, ventilator etc.
- Virtual-virtual: simulation of artificial intelligence, relations between the virtual elements create new virtual situations.
Exploration: all 4 combinations should be explored and brought into easily recognizable scenarios

Seamless contextualization and story telling between the worlds (see scenarios)
Elements from the real and virtual world form an ensemble to tell stories or contribute to a common expression.
Exploration: composition of time, space and narrative related mixed AR-media