Surgical Endoscopy

, 24:79 | Cite as

Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system

  • Michael Bajka
  • Stefan Tuchschmid
  • Daniel Fink
  • Gábor Székely
  • Matthias Harders
Article

Abstract

Background

The aims of this study are to determine construct validity for the HystSim virtual-reality (VR) training simulator for hysteroscopy via a new multimetric scoring system (MMSS) and to explore learning curves for both novices and experienced surgeons.

Methods

Fifteen relevant metrics had been identified for diagnostic hysteroscopy by means of hierarchical task decomposition. They were grouped into four modules (visualization, ergonomics, safety, and fluid handling) and individually weighted, building the MMSS for this study. In a first step, 24 novice medical students and 12 experienced gynecologists went through a self-paced teaching tutorial, in which all participants received clearly stated goals and instructions on how to carry out hysteroscopic procedures properly for this study. All subjects performed five repeated trials on two different exercises on HystSim (exploration and diagnosis exercises). After each trial the results were presented to the participants in the form of an automated objective feedback report (AOFR). Construct validity for the MMSS and learning curves were investigated by comparing the performance between novices and experienced surgeons and in between the repeated trials. To study the effect of repeated practice, 23 of the novices returned 2 weeks later for a second training session.

Results

Comparing novices with the experienced group, the ergonomics and fluid handling modules resulted in construct validity, while the visualization module did not, and for the safety module the experienced group even scored significantly lower than novices in both exercises. The overall score showed only construct validity when the safety module was excluded. Concerning learning curves, all subjects improved significantly during the training on HystSim, with clear indication that the second training session was beneficial for novice surgeons.

Conclusions

Construct validity for HystSim has been established for different modules of VR metrics on a new MMSS developed for diagnostic hysteroscopy. Careful refinement and further testing of metrics and scores is required before using them as assessment tools for operative skills.

Keywords

Virtual reality Training Simulation Hysteroscopy Evaluation Construct validity Learning curves 

References

  1. 1.
    Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A (2007) Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg 246:771–779CrossRefPubMedGoogle Scholar
  2. 2.
    Khalifa YM, Bogorad D, Gibson V, Peifer J, Nussbaum J (2006) Virtual reality in ophthalmology training. Surv Ophthalmol 51:259–273CrossRefPubMedGoogle Scholar
  3. 3.
    Michelson JD (2006) Simulation in orthopaedic education: an overview of theory and practice. J Bone Joint Surg Am 88:1405–1411CrossRefPubMedGoogle Scholar
  4. 4.
    Panait L, Bell RL, Roberts KE, Duffy AJ (2008) Designing and validating a customized virtual reality-based laparoscopic skills curriculum. J Surg Educ 65:413–417CrossRefPubMedGoogle Scholar
  5. 5.
    Park J, MacRae H, Musselman LJ, Rossos P, Hamstra SJ, Wolman S, Reznick RK (2007) Randomized controlled trial of virtual reality simulator training: transfer to live patients. Am J Surg 194:205–211CrossRefPubMedGoogle Scholar
  6. 6.
    Wignall GR, Denstedt JD, Preminger GM, Cadeddu JA, Pearle MS, Sweet RM, McDougall EM (2008) Surgical simulation: a urological perspective. J Urol 179:1690–1699CrossRefPubMedGoogle Scholar
  7. 7.
    Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463CrossRefPubMedGoogle Scholar
  8. 8.
    Schijven MP, Jakimowicz JJ, Broeders IA, Tseng LN (2005) The Eindhoven laparoscopic cholecystectomy training course—improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc 19:1220–1226CrossRefPubMedGoogle Scholar
  9. 9.
    Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, Jakimowicz JJ (2005) Consensus guidelines for validation of virtual reality surgical simulators. Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes. Surg Endosc 19:1523–1532CrossRefPubMedGoogle Scholar
  10. 10.
    Bajka M, Tuchschmid S, Streich M, Fink D, Székely G, Harders M (2008) Evaluation of a new virtual-reality training simulator for hysteroscopy. Surg Endosc 2008 Apr 24 [Epub ahead of print]Google Scholar
  11. 11.
    Harders M, Bachofen D, Bajka M, Grassi M, Heidelberger B, Sierra R, Spaelter U, Steinemann D, Teschner M, Tuchschmid S, Zatonyi J, Szekely G (2008) Virtual reality based simulation of hysteroscopic interventions. Presence-Teleop Virt Environ 17(5):441–462Google Scholar
  12. 12.
    Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos P, Fowler DL (2005) Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc 19:401–405CrossRefPubMedGoogle Scholar
  13. 13.
    Eriksen JR, Grantcharov T (2005) Objective assessment of laparoscopic skills using a virtual reality stimulator. Surg Endosc 19:1216–1219CrossRefPubMedGoogle Scholar
  14. 14.
    Gallagher AG, Richie K, McClure N, McGuigan J (2001) Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality. World J Surg 25:1478–1483CrossRefPubMedGoogle Scholar
  15. 15.
    McCloy R, Stone R (2001) Science, medicine, and the future. Virtual reality in surgery. BMJ 323:912–915CrossRefPubMedGoogle Scholar
  16. 16.
    Satava RM (1993) Virtual reality surgical simulator. The first steps. Surg Endosc 7:203–205CrossRefPubMedGoogle Scholar
  17. 17.
    Schijven M, Jakimowicz J (2003) Construct validity: experts and novices performing on the Xitact LS500 laparoscopy simulator. Surg Endosc 17:803–810CrossRefPubMedGoogle Scholar
  18. 18.
    Sherman V, Feldman LS, Stanbridge D, Kazmi R, Fried GM (2005) Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator. Surg Endosc 19:678–682CrossRefPubMedGoogle Scholar
  19. 19.
    Taffinder N, Sutton C, Fishwick RJ, McManus IC, Darzi A (1998) Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomised controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform 50:124–130PubMedGoogle Scholar
  20. 20.
    Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills Learning curves and reliability measures. Surg Endosc 16(12):1746–1752CrossRefPubMedGoogle Scholar
  21. 21.
    Moorthy K, Munz Y, Dosis A, Bello F, Darzi A (2003) Motion analysis in the training and assessment of minimally invasive surgery. Minim Invasive Ther Allied Technol 12:137–142PubMedGoogle Scholar
  22. 22.
    Ritter EM, McClusky DA, Gallagher AG, Smith CD (2005) Real-time objective assessment of knot quality with a portable tensiometer is superior to execution time for assessment of laparoscopic knot-tying performance. Surg Innov 12:233–237CrossRefPubMedGoogle Scholar
  23. 23.
    Rosen J, Hannaford B, Richards CG, Sinanan MN (2001) Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills. IEEE Trans Biomed Eng 48:579–591CrossRefPubMedGoogle Scholar
  24. 24.
    Rosen J, Brown JD, Chang L, Sinanan MN, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete markov model. IEEE Trans Biomed Eng 53:399–413CrossRefPubMedGoogle Scholar
  25. 25.
    Van Dongen KW, Tournoij E, van der Zee DC, Schijven MP, Broeders IA (2007) Construct validity of the LapSim: can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc 21:1413–1417CrossRefPubMedGoogle Scholar
  26. 26.
    Mackay S, Datta V, Chang A, Shah J, Kneebone R, Darzi A (2003) Multiple Objective Measures of Skill (MOMS): a new approach to the assessment of technical ability in surgical trainees. Ann Surg 238:291–300PubMedGoogle Scholar
  27. 27.
    Heinrichs WL, Lukoff B, Youngblood P, Dev P, Shavelson R, Hasson HM, Satava RM, McDougall EM, Wetter PA (2007) Criterion-based training with surgical simulators: proficiency of experienced surgeons. JSLS 11:273–302PubMedGoogle Scholar
  28. 28.
    Cao CG, MacKenzie CL, Ibbotson JA, Turner LJ, Blair NP, Nagy AG (1999) Hierarchical decomposition of laparoscopic procedures. Stud Health Technol Inform 62:83–89PubMedGoogle Scholar
  29. 29.
    Tuchschmid S, Bajka M, Bachofen D, Székely G, Harders M (2007) Objective surgical performance assessment for virtual hysteroscopy. Stud Health Technol Inform 125:473–478PubMedGoogle Scholar
  30. 30.
    Gallagher AG, Ritter EM, Satava RM (2003) Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17:1525–1529CrossRefPubMedGoogle Scholar
  31. 31.
    Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241:364–372CrossRefPubMedGoogle Scholar
  32. 32.
    Schijven MP, Jakimowicz J (2004) The learning curve on the Xitact LS 500 laparoscopy simulator: profiles of performance. Surg Endosc 18:121–127CrossRefPubMedGoogle Scholar
  33. 33.
    Hassan I, Weyers P, Maschuw K, Dick B, Gerdes B, Rothmund M, Zielke A (2006) Negative stress-coping strategies among novices in surgery correlate with poor virtual laparoscopic performance. Br J Surg 93:1554–1559CrossRefPubMedGoogle Scholar
  34. 34.
    Aggarwal R, Grantcharov TP, Eriksen JR, Blirup D, Kristiansen VB, Funch-Jensen P, Darzi A (2006) An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 244:310–314CrossRefPubMedGoogle Scholar
  35. 35.
    Chaudhry A, Sutton C, Wood J, Stone R, McCloy R (1999) Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl 81:281–286PubMedGoogle Scholar
  36. 36.
    Gor M, McCloy R, Stone R, Smith A (2003) Virtual reality laparoscopic simulator for assessment in gynaecology. BJOG 110:181–187CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Michael Bajka
    • 1
  • Stefan Tuchschmid
    • 2
  • Daniel Fink
    • 1
  • Gábor Székely
    • 2
  • Matthias Harders
    • 2
  1. 1.Division of Gynaecology, Department of OB/GYNUniversity Hospital ZurichZurichSwitzerland
  2. 2.ETH ZurichZurichSwitzerland

Personalised recommendations