Intelligent Virtual Environments for Surgical Training

Intelligent Virtual Environments for Surgical Training

Mahidol – Bremen Medical Informatics Research Unit, Faculty of Information and Communication Technology, Mahidol University

Title :

Intelligent Virtual Environments for Surgical Training

Researchers :

Prof.Dr.Peter Haddawy (Faculty of ICT, Mahidol University)
Prof.Dr.Siriwan Suebnukarn (Faculty of Dentistry, Thammasat University)
Dr.Phattanapon Rhienmora (School of Science and Technology, Bangkok University)
Dr.Myat Su Yin (Faculty of ICT, Mahidol University)
Ms.Narumol Vannaprathip (Faculty of ICT, Mahidol University)
Mr.Nat Sararit (Faculty of ICT, Mahidol University)
Prof.Dr.Christian Freksa (Bremen Spatial Cognition Center, University of Bremen)
Dr.Holger Schultheis (Bremen Spatial Cognition Center, University of Bremen)
Prof.Dr.Gabriel Zachmann (Computer Graphics and Virtual Reality Lab, University of Bremen)
Prof.Dr.Michael Beetz (Institute for Artificial Intelligence, University of Bremen)

The objective of this project is to explore the potential of the integration of techniques from Virtual Reality (VR) and Artificial Intelligence to increase the effectiveness of surgical training by providing personalized instruction, assessment, and formative feedback in a form and on a scale not possible in the physical world. Work is being carried out in the domain of dental surgery. The systems being built promise to provide dental students with an increased amount of guided practice at relatively low cost.

The following systems have been implemented:
• An immersive virtual reality dental simulator that provides a 3D stereoscopic visualization of the entire dental operating room environment. Using two haptic devices the simulator allows a user to manipulate the dental instrument and the mirror and renders results graphically and haptically. Throughout the procedure, the simulator gathers kinematic data that describes how the procedure was carried out. Analytical routines identify errors in the outcome and their causes in the procedure. Formative feedback is delivered using a variety of modalities. One is a video replay of the procedure, highlighting the errors as they occur. Another is a haptic feedback to teach students the proper use of force.
• An intelligent tutoring system for teaching decision making skills in dental surgery. This work has a particular focus on teaching situation awareness skills in decision making, which no other existing surgical simulator does.
• An immersive VR simulator for teaching emergency management in dental surgery. This simulator uses low-cost readily available VR technology to maximize accessibility to students. In this way students can practice anytime and anywhere.

Numerous clinical studies demonstrating the effectiveness of these systems have been published.

Award Grant related to the Project : Fellowship and grant from Hanse-Wissenschaftskolleg Institute for Advanced Study; Grant from Thailand Research Fund

Publishing :
• N. Vannaprathip, P. Haddawy, H. Schultheis, S. Suebnukarn, P. Limsuvan, A. Intaraudom, N. Aiemlaor, C. Teemuenvai, A Planning-Based Approach to Generating Tutorial Dialog for Teaching Surgical Decision Making, Proc. 14th Int’l Conf. on Intelligent Tutoring Systems, Montreal, 11-15 June 2018.
• M. Su Yin, P. Haddawy, S. Suebnukarn, P. Rhienmora, Automated Outcome Scoring in a Virtual Reality Simulator for Endodontic Surgery, Computer Methods and Programs in Biomedicine, 153, pp 53-59, 2018.
• Dwisaptarini A, Suebnukarn S, Rhienmora P, Koontongkaew S, Haddawy P. Effectiveness of the multilayered caries model and visuo-tactile virtual reality simulator for minimally invasive caries removal: A randomized controlled trial. Operative Dentistry, 43(3), pp E110 – E118, May/June 2018.
• N. Vannaprathip, P. Haddawy, H. Schultheis, S. Suebnukarn, P. Limsuvan, A. Intaraudom, N. Aiemlaor, C. Teemuenvai, A Planning-Based Approach to Generating Tutorial Dialog for Teaching Surgical Decision Making, Proc. 14th Int’l Conf. on Intelligent Tutoring Systems, Montreal, June 2018.
• M. Su Yin, P. Haddawy, S. Suebnukarn, H. Schultheis, Use of Haptic Feedback to Train Correct Application of Force in Endodontic Surgery, Proc. 22nd ACM Int’l Conf. on Intelligent User Interfaces (IUI 2017), Limassol, March 2017.
• N. Sararit, P. Hadddawy, S. Suebnukarn, A VR Simulator for Emergency Management in Endodontic Surgery, Proc. 22nd ACM Int’l Conf. on Intelligent User Interfaces (IUI 2017), Limassol, March 2017.
• N. Vannaprathip, P. Haddawy, S. Suebnukarn, P. Sangsartra, N. Sasikhant, S. Sangutai, Desitra: A Simulator for Teaching Situated Decision Making in Dental Surgery, Proc. 21st ACM Int’l Conf. on Intelligent User Interfaces (IUI 2016), Sonoma, March 2016.
• M. Su Yin, P. Haddawy, S. Suebnukarn, P. Rhienmora, Toward Intelligent Tutorial Feedback in Surgical Simulation: Robust Outcome Scoring for Endodontic Surgery, Proc. 21st ACM Int’l Conf. on Intelligent User Interfaces (IUI 2016), Sonoma, March 2016.

Key Contact Person :
Prof.Dr.Peter Haddawy
Faculty of ICT, Mahidol University
peter.had@mahidol.ac.th