Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system
The aims of this study are to determine construct validity for the HystSim virtual-reality (VR) training simulator for hysteroscopy via a new multimetric scoring system (MMSS) and to explore learning curves for both novices and experienced surgeons.
Fifteen relevant metrics had been identified for diagnostic hysteroscopy by means of hierarchical task decomposition. They were grouped into four modules (visualization, ergonomics, safety, and fluid handling) and individually weighted, building the MMSS for this study. In a first step, 24 novice medical students and 12 experienced gynecologists went through a self-paced teaching tutorial, in which all participants received clearly stated goals and instructions on how to carry out hysteroscopic procedures properly for this study. All subjects performed five repeated trials on two different exercises on HystSim (exploration and diagnosis exercises). After each trial the results were presented to the participants in the form of an automated objective feedback report (AOFR). Construct validity for the MMSS and learning curves were investigated by comparing the performance between novices and experienced surgeons and in between the repeated trials. To study the effect of repeated practice, 23 of the novices returned 2 weeks later for a second training session.
Comparing novices with the experienced group, the ergonomics and fluid handling modules resulted in construct validity, while the visualization module did not, and for the safety module the experienced group even scored significantly lower than novices in both exercises. The overall score showed only construct validity when the safety module was excluded. Concerning learning curves, all subjects improved significantly during the training on HystSim, with clear indication that the second training session was beneficial for novice surgeons.
Construct validity for HystSim has been established for different modules of VR metrics on a new MMSS developed for diagnostic hysteroscopy. Careful refinement and further testing of metrics and scores is required before using them as assessment tools for operative skills.
KeywordsVirtual reality Training Simulation Hysteroscopy Evaluation Construct validity Learning curves
- 8.Schijven MP, Jakimowicz JJ, Broeders IA, Tseng LN (2005) The Eindhoven laparoscopic cholecystectomy training course—improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc 19:1220–1226CrossRefPubMedGoogle Scholar
- 9.Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, Jakimowicz JJ (2005) Consensus guidelines for validation of virtual reality surgical simulators. Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes. Surg Endosc 19:1523–1532CrossRefPubMedGoogle Scholar
- 10.Bajka M, Tuchschmid S, Streich M, Fink D, Székely G, Harders M (2008) Evaluation of a new virtual-reality training simulator for hysteroscopy. Surg Endosc 2008 Apr 24 [Epub ahead of print]Google Scholar
- 11.Harders M, Bachofen D, Bajka M, Grassi M, Heidelberger B, Sierra R, Spaelter U, Steinemann D, Teschner M, Tuchschmid S, Zatonyi J, Szekely G (2008) Virtual reality based simulation of hysteroscopic interventions. Presence-Teleop Virt Environ 17(5):441–462Google Scholar