WORKSHOPS

With the introduction of affordable, high-quality and increasingly wireless Head-Mounted Displays (HMDs), Virtual Reality (VR) became a multimedia powerhouse driving futuristic visions like the metaverse. There is less time needed to think about tracking of controllers or performance restrictions, the basics of VR are provided more and more effortlessly. In combination with the widespread adoption, this opens new possibilities for researchers working on different frontiers.

In the area of Serious VR Simulations, often called Digital Twins, addon cameras allow for Mixed Reality (MR) concepts, fusing together reality with virtuality by e.g., integrating physical, tangible objects into the traditionally purely virtual world.
The previously clunky VR Serious Games genre profits from high-quality serious content, which can be broadly experienced to enhance immersion and knowledge transfer.

In both areas, the full potential of this vision-based technology is still limited through the lack of framework support and tangible controller feedback for otherwise purely virtual objects. We want to bring together experts from different computing and engineering disciplines to overcome the barriers of complex VR simulations and games in two research directions:

Serious VR Simulations range from traditional replicas of environments and study scenarios to better understand the behavior of people, teach skills for specific tasks or operate complex remote technology like robots. Over the last years, new user groups for simulated environments, such as users of remote teaching tools during the COIVID-19 pandemic or elderly people who want to have an immersive traveling experience, have been emerging because of the progress and affordable nature of VR technology.

VR Serious Games combine the world of playing games for immersive entertainment with the goal of learning in an almost subconsciously manner. Combining these two approaches requires meeting the demands of both. Learning content must be presented correctly and coherently, but on the other hand, it must be perfectly integrated into a game concept.

Today, there are still various unsolved and related challenges for Serious VR Simulations and VR Serious Games. Major challenges include, but are not limited to:

• Tangible Objects and alternative Controllers: Traditionally, the included HMD controllers and potentially trackers are mostly used to interact with virtual objects. This limits the possibilities to have rich, multi-dimensional feedback and to enhance physical presence through tangible non-controller-shaped items. Tracking physical objects and including them into the virtual world (Augmented Virtuality) is a challenge with the need for a tracking pipeline. Addon cameras in combination with state-of-the-art algorithms can overcome those limitations. In this workshop we want to explore the realization of such rich and tangible experiences.

• VR Multiplayer Framework: Reliably connecting users and supporting versatility in terms of platforms (including microcontrollers for smart items), is still an open issue. In a game engine such as Unity, there is up until this day a limitation in supporting self-made tracking algorithms, e.g., Machine Learning-based ones. On the other hand, game engines provide the important capability to efficiently render virtual content and provide the central game logic. We want to propose building blocks to solve such limitations and bring together mobile and microcontroller platforms into a framework.
• VR Serious Games: Function based on an immersive learning environment. The virtual world can adapt to the learning content. For example, let the user travel virtually to distant places or offer the possibility of virtual time travel to specific points in time or events in history. In VENUS we want to explore new possibilities for Serious Games in VR.

As organizers we will contribute to the workshop by presenting research projects in both directions: Serious VR Simulations and VR Serious Games.

A realistic replica of a supermarket has been created to evaluate smartphone-based health apps in a standardized environment with the help of tracked markers which allow the inclusion of the display in VR. Additionally, some controller designs with multi-dimensional feedback will be displayed, e.g., for sports simulations. We present the utilization of VR in Serious Games in various context directions with developed examples such as cultural heritage. To provide a basic structure, we describe challenges we faced in a Positioning Paper, which is presented together with solutions we found to spark an active discussion with the participants. Further research is included through the submitted papers which have been reviewed to fit in the scope of the workshop: