KnotbAR: Surgical Knot Tying XR
The project Tangible Reality - Interaction of Hands with Physical Tools in Mixed Reality investigates new concepts to integrate the user’s hands into virtual reality applications to improve the interaction with physical objects. The project developed a mixed reality prototype to facilitate the acquisition of practical skills for surgical knot tying. By wearing a virtual reality headset combined with a passthrough function, users receive visual instructions that adapt to their hand movement. The system can measure and display the thread tension and tensile strength of the knot, adapt the learning speed to the user’s progress and provide individualized feedback based on hand and finger tracking. We found evidence that spatial and tangible interaction in mixed reality systems helps trainees to acquire practical skills in medical training and supports the transition from theory to practice.
Info
- Project lead: Prof. Dr. Moritz Queisner
- Team: Christopher Remde, Prof. Dr. Igor Sauer
- Duration 2021 - 2025
- Funding: Federal Ministry of Education and Research (BMBF), Interactive Systems in Virtual and Real Environments program
Partners
- Charité –Universitätsmedizin Berlin, Experimental Surgery, Digital Surgery Lab
- Universität Passau, Lehrstuhl für Psychologie mit Schwerpunkt Mensch-Maschine-Interaktion
- German Research Center for Artificial Intelligence
- NMY Mixed-Reality Communication GmbH