Skip to main content
SpringerLink
Log in

Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

The aims of this study are to determine construct validity for the HystSim virtual-reality (VR) training simulator for hysteroscopy via a new multimetric scoring system (MMSS) and to explore learning curves for both novices and experienced surgeons.

Methods

Fifteen relevant metrics had been identified for diagnostic hysteroscopy by means of hierarchical task decomposition. They were grouped into four modules (visualization, ergonomics, safety, and fluid handling) and individually weighted, building the MMSS for this study. In a first step, 24 novice medical students and 12 experienced gynecologists went through a self-paced teaching tutorial, in which all participants received clearly stated goals and instructions on how to carry out hysteroscopic procedures properly for this study. All subjects performed five repeated trials on two different exercises on HystSim (exploration and diagnosis exercises). After each trial the results were presented to the participants in the form of an automated objective feedback report (AOFR). Construct validity for the MMSS and learning curves were investigated by comparing the performance between novices and experienced surgeons and in between the repeated trials. To study the effect of repeated practice, 23 of the novices returned 2 weeks later for a second training session.

Results

Comparing novices with the experienced group, the ergonomics and fluid handling modules resulted in construct validity, while the visualization module did not, and for the safety module the experienced group even scored significantly lower than novices in both exercises. The overall score showed only construct validity when the safety module was excluded. Concerning learning curves, all subjects improved significantly during the training on HystSim, with clear indication that the second training session was beneficial for novice surgeons.

Conclusions

Construct validity for HystSim has been established for different modules of VR metrics on a new MMSS developed for diagnostic hysteroscopy. Careful refinement and further testing of metrics and scores is required before using them as assessment tools for operative skills.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Aggarwal R, Ward J, Balasundaram I, Sains P, Athanasiou T, Darzi A (2007) Proving the effectiveness of virtual reality simulation for training in laparoscopic surgery. Ann Surg 246:771–779

    Article  PubMed  Google Scholar 

  2. Khalifa YM, Bogorad D, Gibson V, Peifer J, Nussbaum J (2006) Virtual reality in ophthalmology training. Surv Ophthalmol 51:259–273

    Article  PubMed  Google Scholar 

  3. Michelson JD (2006) Simulation in orthopaedic education: an overview of theory and practice. J Bone Joint Surg Am 88:1405–1411

    Article  PubMed  Google Scholar 

  4. Panait L, Bell RL, Roberts KE, Duffy AJ (2008) Designing and validating a customized virtual reality-based laparoscopic skills curriculum. J Surg Educ 65:413–417

    Article  PubMed  Google Scholar 

  5. Park J, MacRae H, Musselman LJ, Rossos P, Hamstra SJ, Wolman S, Reznick RK (2007) Randomized controlled trial of virtual reality simulator training: transfer to live patients. Am J Surg 194:205–211

    Article  PubMed  Google Scholar 

  6. Wignall GR, Denstedt JD, Preminger GM, Cadeddu JA, Pearle MS, Sweet RM, McDougall EM (2008) Surgical simulation: a urological perspective. J Urol 179:1690–1699

    Article  PubMed  Google Scholar 

  7. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463

    Article  PubMed  Google Scholar 

  8. Schijven MP, Jakimowicz JJ, Broeders IA, Tseng LN (2005) The Eindhoven laparoscopic cholecystectomy training course—improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc 19:1220–1226

    Article  CAS  PubMed  Google Scholar 

  9. Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, Jakimowicz JJ (2005) Consensus guidelines for validation of virtual reality surgical simulators. Work Group for Evaluation and Implementation of Simulators and Skills Training Programmes. Surg Endosc 19:1523–1532

    Article  CAS  PubMed  Google Scholar 

  10. Bajka M, Tuchschmid S, Streich M, Fink D, Székely G, Harders M (2008) Evaluation of a new virtual-reality training simulator for hysteroscopy. Surg Endosc 2008 Apr 24 [Epub ahead of print]

  11. Harders M, Bachofen D, Bajka M, Grassi M, Heidelberger B, Sierra R, Spaelter U, Steinemann D, Teschner M, Tuchschmid S, Zatonyi J, Szekely G (2008) Virtual reality based simulation of hysteroscopic interventions. Presence-Teleop Virt Environ 17(5):441–462

    Google Scholar 

  12. Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos P, Fowler DL (2005) Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc 19:401–405

    Article  CAS  PubMed  Google Scholar 

  13. Eriksen JR, Grantcharov T (2005) Objective assessment of laparoscopic skills using a virtual reality stimulator. Surg Endosc 19:1216–1219

    Article  CAS  PubMed  Google Scholar 

  14. Gallagher AG, Richie K, McClure N, McGuigan J (2001) Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality. World J Surg 25:1478–1483

    Article  CAS  PubMed  Google Scholar 

  15. McCloy R, Stone R (2001) Science, medicine, and the future. Virtual reality in surgery. BMJ 323:912–915

    Article  CAS  PubMed  Google Scholar 

  16. Satava RM (1993) Virtual reality surgical simulator. The first steps. Surg Endosc 7:203–205

    Article  CAS  PubMed  Google Scholar 

  17. Schijven M, Jakimowicz J (2003) Construct validity: experts and novices performing on the Xitact LS500 laparoscopy simulator. Surg Endosc 17:803–810

    Article  CAS  PubMed  Google Scholar 

  18. Sherman V, Feldman LS, Stanbridge D, Kazmi R, Fried GM (2005) Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator. Surg Endosc 19:678–682

    Article  CAS  PubMed  Google Scholar 

  19. Taffinder N, Sutton C, Fishwick RJ, McManus IC, Darzi A (1998) Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomised controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform 50:124–130

    CAS  PubMed  Google Scholar 

  20. Gallagher AG, Satava RM (2002) Virtual reality as a metric for the assessment of laparoscopic psychomotor skills Learning curves and reliability measures. Surg Endosc 16(12):1746–1752

    Article  CAS  PubMed  Google Scholar 

  21. Moorthy K, Munz Y, Dosis A, Bello F, Darzi A (2003) Motion analysis in the training and assessment of minimally invasive surgery. Minim Invasive Ther Allied Technol 12:137–142

    PubMed  Google Scholar 

  22. Ritter EM, McClusky DA, Gallagher AG, Smith CD (2005) Real-time objective assessment of knot quality with a portable tensiometer is superior to execution time for assessment of laparoscopic knot-tying performance. Surg Innov 12:233–237

    Article  PubMed  Google Scholar 

  23. Rosen J, Hannaford B, Richards CG, Sinanan MN (2001) Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills. IEEE Trans Biomed Eng 48:579–591

    Article  CAS  PubMed  Google Scholar 

  24. Rosen J, Brown JD, Chang L, Sinanan MN, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete markov model. IEEE Trans Biomed Eng 53:399–413

    Article  PubMed  Google Scholar 

  25. Van Dongen KW, Tournoij E, van der Zee DC, Schijven MP, Broeders IA (2007) Construct validity of the LapSim: can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc 21:1413–1417

    Article  PubMed  Google Scholar 

  26. Mackay S, Datta V, Chang A, Shah J, Kneebone R, Darzi A (2003) Multiple Objective Measures of Skill (MOMS): a new approach to the assessment of technical ability in surgical trainees. Ann Surg 238:291–300

    PubMed  Google Scholar 

  27. Heinrichs WL, Lukoff B, Youngblood P, Dev P, Shavelson R, Hasson HM, Satava RM, McDougall EM, Wetter PA (2007) Criterion-based training with surgical simulators: proficiency of experienced surgeons. JSLS 11:273–302

    PubMed  Google Scholar 

  28. Cao CG, MacKenzie CL, Ibbotson JA, Turner LJ, Blair NP, Nagy AG (1999) Hierarchical decomposition of laparoscopic procedures. Stud Health Technol Inform 62:83–89

    CAS  PubMed  Google Scholar 

  29. Tuchschmid S, Bajka M, Bachofen D, Székely G, Harders M (2007) Objective surgical performance assessment for virtual hysteroscopy. Stud Health Technol Inform 125:473–478

    PubMed  Google Scholar 

  30. Gallagher AG, Ritter EM, Satava RM (2003) Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17:1525–1529

    Article  CAS  PubMed  Google Scholar 

  31. Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD, Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann Surg 241:364–372

    Article  PubMed  Google Scholar 

  32. Schijven MP, Jakimowicz J (2004) The learning curve on the Xitact LS 500 laparoscopy simulator: profiles of performance. Surg Endosc 18:121–127

    Article  CAS  PubMed  Google Scholar 

  33. Hassan I, Weyers P, Maschuw K, Dick B, Gerdes B, Rothmund M, Zielke A (2006) Negative stress-coping strategies among novices in surgery correlate with poor virtual laparoscopic performance. Br J Surg 93:1554–1559

    Article  CAS  PubMed  Google Scholar 

  34. Aggarwal R, Grantcharov TP, Eriksen JR, Blirup D, Kristiansen VB, Funch-Jensen P, Darzi A (2006) An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 244:310–314

    Article  PubMed  Google Scholar 

  35. Chaudhry A, Sutton C, Wood J, Stone R, McCloy R (1999) Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl 81:281–286

    CAS  PubMed  Google Scholar 

  36. Gor M, McCloy R, Stone R, Smith A (2003) Virtual reality laparoscopic simulator for assessment in gynaecology. BJOG 110:181–187

    Article  PubMed  Google Scholar 

Download references

Acknowledgment

The authors would like to thank all developers of the hysteroscopy simulator project. This research has been supported by the NCCR Co-Me of the Swiss National Science Foundation. Funding: NCCR Co-Me (Computer Aided and Image Guided Medical Interventions) of the Swiss National Science Foundation (http://www.co-me.ch).

Author information

Authors and Affiliations

  1. Division of Gynaecology, Department of OB/GYN, University Hospital Zurich, Zurich, 8091, Switzerland

    Michael Bajka & Daniel Fink

  2. ETH Zurich, Zurich, 8092, Switzerland

    Stefan Tuchschmid, Gábor Székely & Matthias Harders

Authors
  1. Michael Bajka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Tuchschmid
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Fink
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gábor Székely
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Matthias Harders
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Bajka.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bajka, M., Tuchschmid, S., Fink, D. et al. Establishing construct validity of a virtual-reality training simulator for hysteroscopy via a multimetric scoring system. Surg Endosc 24, 79–88 (2010). https://doi.org/10.1007/s00464-009-0582-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-009-0582-4

Keywords

Search

Navigation