XR Instrument Training
The project Tangible Reality - Interaction of Hands with Physical Tools in Mixed Reality explores new approaches to incorporating users’ hands into virtual reality applications to improve interaction with physical objects. The primary objective is to design new digital learning and training scenarios that effectively integrate theoretical learning with practical application. Using the OTA Tool Picker learning application, trainees can explore the arrangement and functions of surgical instruments on an operating table. The app links image-based instructions to real-world tools and actions. Unlike traditional learning methods, such as animations, illustrations or videos, this approach allows learners to engage with work scenarios from a first-person perspective, at full scale, and with direct physical interaction. By bridging the gap between theoretical knowledge and hands-on practice, mixed reality presentations enable more effective skill acquisition. Through embodied perception and interaction with surgical instruments, learners can transfer knowledge into practical skills more seamlessly. Automated feedback on tasks further supports the development of these skills in a structured, controlled environment. Preliminary findings suggest that spatial and tangible interaction in mixed reality systems enhances the acquisition of practical skills in medical training, facilitating the transition from theoretical understanding to practical application.
Info
- Project lead: Prof. Dr. Moritz Queisner
- Team: Christopher Remde, Prof. Dr. Igor Sauer
- Duration 2021 - 2025
- Funding: Federal Ministry of Education and Research (BMBF), Interactive Systems in Virtual and Real Environments program
- Further info
Partners
- Charité –Universitätsmedizin Berlin, Experimental Surgery, Digital Surgery Lab
- Universität Passau, Arbeitspsychologie
- German Research Center for Artificial Intelligence (DKFZ)
- NMY Mixed-Reality Communication GmbH
- Berliner Bildungscampus für Gesundheitsberufe – BBG
XR Instrument Training
The project Tangible Reality - Interaction of Hands with Physical Tools in Mixed Reality explores new approaches to incorporating users’ hands into virtual reality applications to improve interaction with physical objects. The primary objective is to design new digital learning and training scenarios that effectively integrate theoretical learning with practical application. Using the OTA Tool Picker learning application, trainees can explore the arrangement and functions of surgical instruments on an operating table. The app links image-based instructions to real-world tools and actions. Unlike traditional learning methods, such as animations, illustrations or videos, this approach allows learners to engage with work scenarios from a first-person perspective, at full scale, and with direct physical interaction. By bridging the gap between theoretical knowledge and hands-on practice, mixed reality presentations enable more effective skill acquisition. Through embodied perception and interaction with surgical instruments, learners can transfer knowledge into practical skills more seamlessly. Automated feedback on tasks further supports the development of these skills in a structured, controlled environment. Preliminary findings suggest that spatial and tangible interaction in mixed reality systems enhances the acquisition of practical skills in medical training, facilitating the transition from theoretical understanding to practical application.
Info
- Project lead: Prof. Dr. Moritz Queisner
- Team: Christopher Remde, Prof. Dr. Igor Sauer
- Duration 2021 - 2025
- Funding: Federal Ministry of Education and Research (BMBF), Interactive Systems in Virtual and Real Environments program