Skip to main content

Advertisement

SpringerLink
Log in

Training and assessment using the LapSim laparoscopic simulator: a scoping review of validity evidence

  • Review Article
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Introduction

The LapSim (Surgical Science, Sweden) laparoscopic simulator is a high-fidelity virtual reality simulator for use in endoscopic surgical training. This review critiques the current validity evidence for the LapSim laparoscopic simulator, specifically with respect to its potential use as a tool and method of training and assessment in surgery.

Methods

A scoping review of the MEDLINE (PubMed), EMBASE, Cochrane and Web of Science databases was conducted in accordance with PRISMA guidelines (2020)—scoping review extension. Articles were included if they presented validity evidence for the use of the LapSim in operative skill training or assessment, in accordance with Messick’s validity framework. European Association of Endoscopic Surgeons (EAES) guidelines (2005) were used to provide recommendations for the use of the LapSim in operative performance training and assessments.

Results

Forty-nine articles were included. An EAES level 2 recommendation was provided with regard to the internal consistency reliability of automated performance metrics in assessing performance. An EAES recommendation of 2 was awarded with respect to the ability of the LapSim to discriminate based on case volume and overall laparoscopic experience (relationships with other variables). Performance assessment metrics on the LapSim correlate with improved performance in the operating room (EAES level of recommendation 1—consequential validity).

Conclusion

The LapSim has accumulated substantial evidence supporting the validity of its use in surgical training and assessment. Future studies should explore the relationship between the achievement of performance benchmarks on the LapSim and subsequent patient outcomes, and interrogate the benefits of implementing virtual reality simulation training and assessment curricula in post-graduate surgical training programmes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

(Adapted from [63]

Similar content being viewed by others

References

  1. Hong M, Rozenblit JW, Hamilton AJ (2021) Simulation-based surgical training systems in laparoscopic surgery: a current review. Virtual Real 25:491–510

    Article  Google Scholar 

  2. Toale C, Morris M, Kavanagh D (2021) Perceptions and experiences of simulation-based assessment of technical skill in surgery: a scoping review. Am J Surg 222:723–730

    Article  PubMed  Google Scholar 

  3. McDougall EM (2007) Validation of surgical simulators. J Endourol 21:244–247

    Article  PubMed  Google Scholar 

  4. Carter FJ, Schijven MP, Aggarwal R, Grantcharov T, Francis NK, Hanna GB, Jakimowicz JJ (2005) Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc 19:1523–1532

    Article  CAS  PubMed  Google Scholar 

  5. Fairhurst K, Strickland A, Maddern G (2011) The LapSim virtual reality simulator: promising but not yet proven. Surg Endosc 25:343–355

    Article  PubMed  Google Scholar 

  6. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MD, Horsley T, Weeks L (2018) PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 169:467–473

    Article  PubMed  Google Scholar 

  7. Smyer P (2011) The medical education research study quality instrument (MERSQI)

  8. Hambleton RK, Arrasmith G, Sheehan DS, Grobe RP, Hathaway W, Doherty V, Forbes D, Houser R, Ingebo G, Kingsbury G (1986) Standards for educational and psychological testing: six reviews. JSTOR

  9. Cook DA, Lineberry M (2016) Consequences validity evidence: evaluating the impact of educational assessments. Acad Med 91:785–795

    Article  PubMed  Google Scholar 

  10. Cook DA, Hatala R (2016) Validation of educational assessments: a primer for simulation and beyond. Adv Simul 1:31

    Article  Google Scholar 

  11. Doneza JA, Palvia V, Lerner VT, Overbey J, Levie M, Brodman M, Ascher-Walsh C (2021) Correlation of surgical case volume and fellowship training with performance on simulated procedural tasks. Am J Obstet Gynecol 225:548.e541-548.e510

    Article  Google Scholar 

  12. Haidari TA, Bjerrum F, Hansen HJ, Konge L, Petersen RH (2022) Simulation-based VATS resection of the five lung lobes: a technical skills test. Surg Endosc 36:1234–1242

    Article  PubMed  Google Scholar 

  13. Jensen K, Hansen HJ, Petersen RH, Neckelmann K, Vad H, Møller LB, Pedersen JH, Konge L (2019) Evaluating competency in video-assisted thoracoscopic surgery (VATS) lobectomy performance using a novel assessment tool and virtual reality simulation. Surg Endosc 33:1465–1473

    Article  PubMed  Google Scholar 

  14. Våpenstad C, Hofstad EF, Bø LE, Kuhry E, Johnsen G, Mårvik R, Langø T, Hernes TN (2017) Lack of transfer of skills after virtual reality simulator training with haptic feedback. Minim Invasive Ther Allied Technol 26:346–354

    Article  PubMed  Google Scholar 

  15. Mathews S, Brodman M, D’Angelo D, Chudnoff S, McGovern P, Kolev T, Bensinger G, Mudiraj S, Nemes A, Feldman D, Kischak P, Ascher-Walsh C (2017) Predictors of laparoscopic simulation performance among practicing obstetrician gynecologists. Am J Obstet Gynecol 217:596.e591-596.e597

    Article  Google Scholar 

  16. Bjerrum F, Strandbygaard J, Rosthøj S, Grantcharov T, Ottesen B, Sorensen JL (2017) Evaluation of procedural simulation as a training and assessment tool in general surgery-simulating a laparoscopic appendectomy. J Surg Educ 74:243–250

    Article  PubMed  Google Scholar 

  17. Jensen K, Bjerrum F, Hansen HJ, Petersen RH, Pedersen JH, Konge L (2017) Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy. Surg Endosc 31:2520–2528

    Article  PubMed  Google Scholar 

  18. Jensen K, Bjerrum F, Hansen HJ, Petersen RH, Pedersen JH, Konge L (2015) A new possibility in thoracoscopic virtual reality simulation training: development and testing of a novel virtual reality simulator for video-assisted thoracoscopic surgery lobectomy. Interact Cardiovasc Thorac Surg 21:420–426

    Article  PubMed  Google Scholar 

  19. Alwaal A, Al-Qaoud TM, Haddad RL, Alzahrani TM, Delisle J, Anidjar M (2015) Transfer of skills on LapSim virtual reality laparoscopic simulator into the operating room in urology. Urol Ann 7:172–176

    Article  PubMed  PubMed Central  Google Scholar 

  20. Lin D, Pena G, Field J, Altree M, Marlow N, Babidge W, Hewett P, Maddern G (2016) What are the demographic predictors in laparoscopic simulator performance? ANZ J Surg 86:983–989

    Article  PubMed  Google Scholar 

  21. Akdemir A, Sendağ F, Oztekin MK (2014) Laparoscopic virtual reality simulator and box trainer in gynecology. Int J Gynaecol Obstet 125:181–185

    Article  PubMed  Google Scholar 

  22. Khan MW, Lin D, Marlow N, Altree M, Babidge W, Field J, Hewett P, Maddern G (2014) Laparoscopic skills maintenance: a randomized trial of virtual reality and box trainer simulators. J Surg Educ 71:79–84

    Article  PubMed  Google Scholar 

  23. Akdemir A, Ergenoğlu AM, Yeniel A, Sendağ F (2013) Conventional box model training improves laparoscopic skills during salpingectomy on LapSim: a randomized trial. J Turk Ger Gynecol Assoc 14:157–162

    Article  PubMed  PubMed Central  Google Scholar 

  24. Xafis V, Babidge W, Field J, Altree M, Marlow N, Maddern G (2013) The efficacy of laparoscopic skills training in a Mobile Simulation Unit compared with a fixed site: a comparative study. Surg Endosc 27:2606–2612

    Article  PubMed  Google Scholar 

  25. Lehmann KS, Gröne J, Lauscher JC, Ritz JP, Holmer C, Pohlen U, Buhr HJ (2012) Simulation training in surgical education—application of virtual reality laparoscopic simulators in a surgical skills course. Zentralbl Chir 137:130–137

    CAS  PubMed  Google Scholar 

  26. Tan SC, Marlow N, Field J, Altree M, Babidge W, Hewett P, Maddern GJ (2012) A randomized crossover trial examining low- versus high-fidelity simulation in basic laparoscopic skills training. Surg Endosc 26:3207–3214

    Article  PubMed  Google Scholar 

  27. Shetty S, Panait L, Baranoski J, Dudrick SJ, Bell RL, Roberts KE, Duffy AJ (2012) Construct and face validity of a virtual reality-based camera navigation curriculum. J Surg Res 177:191–195

    Article  PubMed  Google Scholar 

  28. Kovac E, Azhar RA, Quirouet A, Delisle J, Anidjar M (2012) Construct validity of the LapSim virtual reality laparoscopic simulator within a urology residency program. Can Urol Assoc J 6:253–259

    Article  PubMed  PubMed Central  Google Scholar 

  29. Palter VN, Graafland M, Schijven MP, Grantcharov TP (2012) Designing a proficiency-based, content validated virtual reality curriculum for laparoscopic colorectal surgery: a Delphi approach. Surgery 151:391–397

    Article  PubMed  Google Scholar 

  30. Våpenstad C, Hofstad E, Bø LE, Chmarra M, Kuhry E, Johnsen G, Mårvik R, Langø T (2012) Limitations of haptic feedback devices on construct validity of the LapSim((R)) virtual reality simulator. Surg Endosc 27

  31. Panait L, Larios JM, Brenes RA, Fancher TT, Ajemian MS, Dudrick SJ, Sanchez JA (2011) Surgical skills assessment of applicants to general surgery residency. J Surg Res 170:189–194

    Article  PubMed  Google Scholar 

  32. Calatayud D, Arora S, Aggarwal R, Kruglikova I, Schulze S, Funch-Jensen P, Grantcharov T (2010) Warm-up in a virtual reality environment improves performance in the operating room. Ann Surg 251:1181–1185

    Article  PubMed  Google Scholar 

  33. van Dongen KW, Ahlberg G, Bonavina L, Carter FJ, Grantcharov TP, Hyltander A, Schijven MP, Stefani A, van der Zee DC, Broeders IA (2011) European consensus on a competency-based virtual reality training program for basic endoscopic surgical psychomotor skills. Surg Endosc 25:166–171

    Article  PubMed  Google Scholar 

  34. Gauger PG, Hauge LS, Andreatta PB, Hamstra SJ, Hillard ML, Arble EP, Kasten SJ, Mullan PB, Cederna PS, Minter RM (2010) Laparoscopic simulation training with proficiency targets improves practice and performance of novice surgeons. Am J Surg 199:72–80

    Article  PubMed  Google Scholar 

  35. Tanoue K, Uemura M, Kenmotsu H, Ieiri S, Konishi K, Ohuchida K, Onimaru M, Nagao Y, Kumashiro R, Tomikawa M, Hashizume M (2010) Skills assessment using a virtual reality simulator, LapSim, after training to develop fundamental skills for endoscopic surgery. Minim Invasive Ther Allied Technol 19:24–29

    Article  PubMed  Google Scholar 

  36. Maagaard M, Sorensen JL, Oestergaard J, Dalsgaard T, Grantcharov TP, Ottesen BS, Larsen CR (2011) Retention of laparoscopic procedural skills acquired on a virtual-reality surgical trainer. Surg Endosc 25:722–727

    Article  PubMed  Google Scholar 

  37. Schreuder HW, van Dongen KW, Roeleveld SJ, Schijven MP, Broeders IA (2009) Face and construct validity of virtual reality simulation of laparoscopic gynecologic surgery. Am J Obstet Gynecol 200:540.e541-548

    Article  Google Scholar 

  38. Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802

    Article  PubMed  PubMed Central  Google Scholar 

  39. Sinha P, Hogle NJ, Fowler DL (2008) Do the laparoscopic skills of trainees deteriorate over time? Surg Endosc 22:2018–2025

    Article  PubMed  Google Scholar 

  40. Hogle NJ, Widmann WD, Ude AO, Hardy MA, Fowler DL (2008) Does training novices to criteria and does rapid acquisition of skills on laparoscopic simulators have predictive validity or are we just playing video games? J Surg Educ 65:431–435

    Article  PubMed  Google Scholar 

  41. Kundhal PS, Grantcharov TP (2009) Psychomotor performance measured in a virtual environment correlates with technical skills in the operating room. Surg Endosc 23:645–649

    Article  PubMed  Google Scholar 

  42. Salgado J, Grantcharov TP, Papasavas PK, Gagne DJ, Caushaj PF (2009) Technical skills assessment as part of the selection process for a fellowship in minimally invasive surgery. Surg Endosc 23:641–644

    Article  PubMed  Google Scholar 

  43. Panait L, Bell RL, Roberts KE, Duffy AJ (2008) Designing and validating a customized virtual reality-based laparoscopic skills curriculum. J Surg Educ 65:413–417

    Article  PubMed  Google Scholar 

  44. Botden SM, Buzink SN, Schijven MP, Jakimowicz JJ (2007) Augmented versus virtual reality laparoscopic simulation: what is the difference? A comparison of the ProMIS augmented reality laparoscopic simulator versus LapSim virtual reality laparoscopic simulator. World J Surg 31:764–772

    Article  PubMed  PubMed Central  Google Scholar 

  45. van Dongen KW, Tournoij E, van der Zee DC, Schijven MP, Broeders IA (2007) Construct validity of the LapSim: can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc 21:1413–1417

    Article  PubMed  Google Scholar 

  46. Newmark J, Dandolu V, Milner R, Grewal H, Harbison S, Hernandez E (2007) Correlating virtual reality and box trainer tasks in the assessment of laparoscopic surgical skills. Am J Obstet Gynecol 197:546.e541–544

    Article  Google Scholar 

  47. Hogle NJ, Briggs WM, Fowler DL (2007) Documenting a learning curve and test-retest reliability of two tasks on a virtual reality training simulator in laparoscopic surgery. J Surg Educ 64:424–430

    Article  PubMed  Google Scholar 

  48. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA 3rd, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804

    Article  PubMed  Google Scholar 

  49. Hassan I, Gerdes B, Koller M, Langer P, Rothmund M, Zielke A (2006) Clinical background is required for optimum performance with a VR laparoscopy simulator. Comput Aided Surg 11:103–106

    Article  CAS  PubMed  Google Scholar 

  50. Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L, Gauger PG, Minter RM (2006) Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 191:28–32

    Article  PubMed  Google Scholar 

  51. Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sørensen JL, Dalsgaard T, Ottesen B (2006) Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 20:1460–1466

    Article  CAS  PubMed  Google Scholar 

  52. Aggarwal R, Grantcharov TP, Eriksen JR, Blirup D, Kristiansen VB, Funch-Jensen P, Darzi A (2006) An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 244:310–314

    Article  PubMed  PubMed Central  Google Scholar 

  53. Aggarwal R, Tully A, Grantcharov T, Larsen CR, Miskry T, Farthing A, Darzi A (2006) Virtual reality simulation training can improve technical skills during laparoscopic salpingectomy for ectopic pregnancy. BJOG 113:1382–1387

    Article  CAS  PubMed  Google Scholar 

  54. Eriksen JR, Grantcharov T (2005) Objective assessment of laparoscopic skills using a virtual reality stimulator. Surg Endosc 19:1216–1219

    Article  CAS  PubMed  Google Scholar 

  55. Hassan I, Sitter H, Schlosser K, Zielke A, Rothmund M, Gerdes B (2005) A virtual reality simulator for objective assessment of surgeons’ laparoscopic skill. Chirurg 76:151–156

    Article  CAS  PubMed  Google Scholar 

  56. Langelotz C, Kilian M, Paul C, Schwenk W (2005) LapSim virtual reality laparoscopic simulator reflects clinical experience in German surgeons. Langenbecks Arch Surg 390:534–537

    Article  CAS  PubMed  Google Scholar 

  57. Sherman V, Feldman LS, Stanbridge D, Kazmi R, Fried GM (2005) Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator. Surg Endosc 19:678–682

    Article  CAS  PubMed  Google Scholar 

  58. Duffy AJ, Hogle NJ, McCarthy H, Lew JI, Egan A, Christos P, Fowler DL (2005) Construct validity for the LAPSIM laparoscopic surgical simulator. Surg Endosc 19:401–405

    Article  CAS  PubMed  Google Scholar 

  59. Hassan I, Sitter H, Schlosser K, Zielke A, Rothmund M, Gerdes B (2005) Laparoskopiesimulator. Chirurg 76:151–156

    Article  CAS  PubMed  Google Scholar 

  60. Cosman PH, Hugh TJ, Shearer CJ, Merrett ND, Biankin AV, Cartmill JA (2007) Skills acquired on virtual reality laparoscopic simulators transfer into the operating room in a blinded, randomised, controlled trial. Stud Health Technol Inform 125:76–81

    CAS  PubMed  Google Scholar 

  61. Mazzone E, Puliatti S, Amato M, Bunting B, Rocco B, Montorsi F, Mottrie A, Gallagher AG (2021) A systematic review and meta-analysis on the impact of proficiency-based progression simulation training on performance outcomes. Ann Surg 274:281–289

    Article  PubMed  Google Scholar 

  62. Zevin B, Aggarwal R, Grantcharov TP (2014) Surgical simulation in 2013: why is it still not the standard in surgical training? J Am Coll Surg 218:294–301

    Article  PubMed  Google Scholar 

  63. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. https://doi.org/10.1136/bmj.n71

Download references

Funding

Financial support was provided as part of a higher degree (C Toale) by the Royal College of Surgeons/Hermitage Medical Clinic Strategic Academic Recruitment (StAR MD) programme.

Author information

Authors and Affiliations

  1. Department of Surgical Affairs, Royal College of Surgeons in Ireland, 121 St Stephen’s Green, Dublin 2, D02 YN77, Ireland

    Conor Toale, Marie Morris & Dara O. Kavanagh

Authors
  1. Conor Toale
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marie Morris
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dara O. Kavanagh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Conor Toale.

Ethics declarations

Disclosures

Mr Conor Toale, Dr Marie Morris and Mr Dara O. Kavanagh have no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 13 kb)

Supplementary file2 (DOCX 25 kb)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Toale, C., Morris, M. & Kavanagh, D.O. Training and assessment using the LapSim laparoscopic simulator: a scoping review of validity evidence. Surg Endosc 37, 1658–1671 (2023). https://doi.org/10.1007/s00464-022-09593-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-022-09593-0

Keywords

Search

Navigation