Skip to main content

Advertisement

SpringerLink
Log in

Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

The societies of thoracic surgery are working to incorporate simulation and competency-based assessment into specialty training. One challenge is the development of a simulation-based test, which can be used as an assessment tool. The study objective was to establish validity evidence for a virtual reality simulator test of a video-assisted thoracoscopic surgery (VATS) lobectomy of a right upper lobe.

Methods

Participants with varying experience in VATS lobectomy were included. They were familiarized with a virtual reality simulator (LapSim®) and introduced to the steps of the procedure for a VATS right upper lobe lobectomy. The participants performed two VATS lobectomies on the simulator with a 5-min break between attempts. Nineteen pre-defined simulator metrics were recorded.

Results

Fifty-three participants from nine different countries were included. High internal consistency was found for the metrics with Cronbach’s alpha coefficient for standardized items of 0.91. Significant test–retest reliability was found for 15 of the metrics (p-values <0.05). Significant correlations between the metrics and the participants VATS lobectomy experience were identified for seven metrics (p-values <0.001), and 10 metrics showed significant differences between novices (0 VATS lobectomies performed) and experienced surgeons (>50 VATS lobectomies performed). A pass/fail level defined as approximately one standard deviation from the mean metric scores for experienced surgeons passed none of the novices (0 % false positives) and failed four of the experienced surgeons (29 % false negatives).

Conclusion

This study is the first to establish validity evidence for a VATS right upper lobe lobectomy virtual reality simulator test. Several simulator metrics demonstrated significant differences between novices and experienced surgeons and pass/fail criteria for the test were set with acceptable consequences. This test can be used as a first step in assessing thoracic surgery trainees’ VATS lobectomy competency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. McGaghie WC (2008) Research opportunities in simulation-based medical education using deliberate practice. Acad Emerg Med 15:995–1001. doi:10.1111/j.1553-2712.2008.00246.x

    Article  PubMed  Google Scholar 

  2. Price J, Naik V, Boodhwani M, Brandys T, Hendry P, Lam BBK (2011) A randomized evaluation of simulation training on performance of vascular anastomosis on a high-fidelity in vivo model: the role of deliberate practice. J Thorac Cardiovasc Surg 142:496–503. doi:10.1016/j.jtcvs.2011.05.015

    Article  PubMed  Google Scholar 

  3. Marshall MB (2012) Simulation for technical skills. J Thorac Cardiovasc Surg 144:S43–S47. doi:10.1016/j.jtcvs.2012.06.004

    Article  PubMed  Google Scholar 

  4. Crochet P, Aggarwal R, Dubb SS, Ziprin P, Rajaretnam N, Grantcharov T, Ericsson KA, Darzi A (2011) Deliberate practice on a virtual reality laparoscopic simulator enhances the quality of surgical technical skills. Ann Surg 253:1216–1222. doi:10.1097/SLA.0b013e3182197016

    Article  PubMed  Google Scholar 

  5. Seymour NE (2008) VR to OR: a review of the evidence that virtual reality simulation improves operating room performance. World J Surg 32:182–188. doi:10.1007/s00268-007-9307-9

    Article  PubMed  Google Scholar 

  6. Schijven MP, Jakimowicz JJ, Broeders IAMJ, Tseng LNL (2005) The Eindhoven laparoscopic cholecystectomy training course—improving operating room performance using virtual reality training: results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc Other Interv Tech 19:1220–1226. doi:10.1007/s00464-004-2240-1

    Article  CAS  Google Scholar 

  7. Jensen K, Bjerrum F, Hansen HJ, Petersen RH, Pedersen JH, Konge L (2015) A new possibility in thoracoscopic virtual reality simulation training: development and testing of a novel virtual reality simulator for video-assisted thoracoscopic surgery lobectomy. Interact Cardiovasc Thorac Surg 21:420–426. doi:10.1093/icvts/ivv183

    Article  PubMed  Google Scholar 

  8. Seymour NE, Røtnes JS (2006) Challenges to the development of complex virtual reality surgical simulations. Surg Endosc Other Interv Tech 20:1774–1777. doi:10.1007/s00464-006-0107-3

    Article  CAS  Google Scholar 

  9. Jensen K, Ringsted C, Hansen HJ, Petersen RH, Konge L (2014) Simulation-based training for thoracoscopic lobectomy: a randomized controlled trial: virtual-reality versus black-box simulation. Surg Endosc Other Interv Tech 28:1821–1829. doi:10.1007/s00464-013-3392-7

    Article  Google Scholar 

  10. Bjerrum F, Sorensen JL, Konge L, Rosthøj S, Lindschou J, Ottesen B, Strandbygaard J (2016) Randomized clinical trial to examine procedure-to-procedure transfer in laparoscopic simulator training. Br J Surg 103:44–50. doi:10.1002/bjs.9966

    Article  CAS  PubMed  Google Scholar 

  11. Carpenter AJ, Yang SC, Uhlig PN, Colson YL (2008) Envisioning simulation in the future of thoracic surgical education. J Thorac Cardiovasc Surg 135:477–484. doi:10.1016/j.jtcvs.2007.12.005

    Article  PubMed  Google Scholar 

  12. Verrier ED (2011) Joint Council on Thoracic Surgical Education: an investment in our future. J Thorac Cardiovasc Surg 141:318–321. doi:10.1016/j.jtcvs.2010.10.013

    Article  PubMed  Google Scholar 

  13. Moon MR (2014) Technical skills assessment in thoracic surgery education: we won’t get fooled again. J Thorac Cardiovasc Surg 148:2497–2498. doi:10.1016/j.jtcvs.2014.09.057

    Article  PubMed  Google Scholar 

  14. Lodge D, Grantcharov T (2011) Training and assessment of technical skills and competency in cardiac surgery. Eur J Cardiothorac Surg 39:287–293. doi:10.1016/j.ejcts.2010.06.035

    Article  PubMed  Google Scholar 

  15. Aggarwal R, Darzi A (2011) Simulation to enhance patient safety: Why aren’t we there yet? Chest 140:854–858. doi:10.1378/chest.11-0728

    Article  PubMed  Google Scholar 

  16. Stefanidis D, Sevdalis N, Paige J, Zevin B, Aggarwal R, Grantcharov T, Jones DB (2015) Simulation in surgery: What’s needed next? Ann Surg 261:846–853. doi:10.1097/SLA.0000000000000826

    Article  PubMed  Google Scholar 

  17. Baumgartner WA (2003) Cardiothoracic surgery: a specialty in transition—Good to great? Ann Thorac Surg 75:1685–1692. doi:10.1016/S0003-4975(03)00538-1

    Article  PubMed  Google Scholar 

  18. Chitwood WR, Spray TL, Feins RH, Mack MJ (2008) Mission critical: thoracic surgery education reform. Ann Thorac Surg 86:1061–1062. doi:10.1016/j.athoracsur.2008.08.047

    Article  PubMed  Google Scholar 

  19. Crawford FA (2003) Presidential address: thoracic surgery education—responding to a changing environment. J Thorac Cardiovasc Surg 126:1235–1242. doi:10.1016/S0022-5223(03)00814-6

    Article  PubMed  Google Scholar 

  20. Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, Euhus DM, Jeyarajah DR, Thompson WM, Jones DB (2000) Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg 191:272–283. doi:10.1016/S1072-7515(00)00339-2

    Article  CAS  PubMed  Google Scholar 

  21. Torkington J, Smith SGT, Rees BI, Darzi A (2001) Skill transfer from virtual reality to a real laparoscopic task. Surg Endosc 15:1076–1079. doi:10.1007/s004640000233

    Article  CAS  PubMed  Google Scholar 

  22. Paisley AM, Baldwin PJ, Paterson-Brown S (2001) Validity of surgical simulation for the assessment of operative skill. Br J Surg 88:1525–1532. doi:10.1046/j.0007-1323.2001.01880.x

    Article  CAS  PubMed  Google Scholar 

  23. Reznick RK, MacRae H (2006) Teaching surgical skills—changes in the wind. N Engl J Med 355:2664–2669. doi:10.1016/S0084-392X(08)70199-0

    Article  CAS  PubMed  Google Scholar 

  24. Lee R, Enter D, Lou X, Feins RH, Hicks GL, Gasparri M, Takayama H, Young JN, Calhoon JH, Crawford FA, Mokadam NA, Fann JI (2013) The joint council on thoracic surgery education coronary artery assessment tool has high interrater reliability. Ann Thorac Surg 95:2064–2070. doi:10.1016/j.athoracsur.2012.10.090

    Article  PubMed  Google Scholar 

  25. Van Bruwaene S, Schijven MP, Miserez M (2014) Assessment of procedural skills using virtual simulation remains a challenge. J Surg Educ 71:654–661. doi:10.1016/j.jsurg.2014.01.005

    Article  PubMed  Google Scholar 

  26. Seagull FJ, Rooney DM (2014) Filling a void: developing a standard subjective assessment tool for surgical simulation through focused review of current practices. Surgery 156:718–722. doi:10.1016/j.surg.2014.04.048

    Article  PubMed  Google Scholar 

  27. Van Hove PD, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987. doi:10.1002/bjs.7115

    Article  PubMed  Google Scholar 

  28. Hansen HJ, Petersen RH, Christensen M (2011) Video-assisted thoracoscopic surgery (VATS) lobectomy using a standardized anterior approach. Surg Endosc Other Interv Tech 25:1263–1269. doi:10.1007/s00464-010-1355-9

    Article  Google Scholar 

  29. Konge L, Ringsted C, Bjerrum F, Tolsgaard MG, Bitsch M, Sørensen JL, Schroeder TV (2015) The simulation centre at Rigshospitalet, Copenhagen, Denmark. J Surg Educ 72:362–365. doi:10.1016/j.jsurg.2014.11.012

    Article  PubMed  Google Scholar 

  30. Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sørensen JL, Dalsgaard T, Ottesen B (2006) Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 20:1460–1466. doi:10.1007/s00464-005-0745-x

    Article  CAS  PubMed  Google Scholar 

  31. Van Dongen KW, Tournoij E, Van Der Zee DC, Schijven MP, Broeders IAMJ (2007) Construct validity of the LapSim: Can the LapSim virtual reality simulator distinguish between novices and experts? Surg Endosc Other Interv Tech 21:1413–1417. doi:10.1007/s00464-006-9188-2

    Article  Google Scholar 

  32. Eriksen JR, Grantcharov T (2005) Objective assessment of laparoscopic skills using a virtual reality stimulator. Surg Endosc 19:1216–1219. doi:10.1007/s00464-004-2154-y

    Article  CAS  PubMed  Google Scholar 

  33. Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I, Farrell TM (2015) Technical skills assessment toolbox: a review using the unitary framework of validity. Ann Surg 261:251–262. doi:10.1097/SLA.0000000000000520

    Article  PubMed  Google Scholar 

  34. Vassiliou MC, Ghitulescu GA, Feldman LS, Stanbridge D, Leffondré K, Sigman HH, Fried GM (2006) The MISTELS program to measure technical skill in laparoscopic surgery: evidence for reliability. Surg Endosc Other Interv Tech 20:744–747. doi:10.1007/s00464-005-3008-y

    Article  CAS  Google Scholar 

  35. Downing SM (2003) Validity: on the meaningful interpretation of assessment data. Med Educ 37:830–837. doi:10.1046/j.1365-2923.2003.01594.x

    Article  PubMed  Google Scholar 

  36. Rivard JD, Vergis AS, Unger BJ, Hardy KM, Andrew CG, Gillman LM, Park J (2014) Construct validity of individual and summary performance metrics associated with a computer-based laparoscopic simulator. Surg Endosc Other Interv Tech 28:1921–1928. doi:10.1007/s00464-013-3414-5

    Article  Google Scholar 

  37. Woodrum DT, Andreatta PB, Yellamanchilli RK, Feryus L, Gauger PG, Minter RM (2006) Construct validity of the LapSim laparoscopic surgical simulator. Am J Surg 191:28–32. doi:10.1016/j.amjsurg.2005.10.018

    Article  PubMed  Google Scholar 

  38. Fried GM, Feldman LS (2008) Objective assessment of technical performance. World J Surg 32:156–160. doi:10.1007/s00268-007-9143-y

    Article  PubMed  Google Scholar 

  39. MacRae HM (2014) Technical skills assessment: time to take it seriously. Dis Colon Rectum 57:141–142. doi:10.1097/DCR.0000000000000032

    Article  PubMed  Google Scholar 

  40. Hogle NJ, Briggs WM, Fowler DL (2007) Documenting a learning curve and test–retest reliability of two tasks on a virtual reality training simulator in laparoscopic surgery. J Surg Educ 64:424–430. doi:10.1016/j.jsurg.2007.08.007

    Article  PubMed  Google Scholar 

  41. Loukas C, Nikiteas N, Schizas D, Lahanas V, Georgiou E (2012) A head-to-head comparison between virtual reality and physical reality simulation training for basic skills acquisition. Surg Endosc 26:2550–2558. doi:10.1007/s00464-012-2230-7

    Article  PubMed  Google Scholar 

  42. Zendejas B, Ruparel RK, Cook DA (2015) Validity evidence for the fundamentals of laparoscopic surgery (FLS) program as an assessment tool: a systematic review. Surg Endosc Other Interv Tech 30:512–520. doi:10.1007/s00464-015-4233-7

    Article  Google Scholar 

  43. Våpenstad C, Buzink SN (2013) Procedural virtual reality simulation in minimally invasive surgery. Surg Endosc 27:364–377. doi:10.1007/s00464-012-2503-1

    Article  PubMed  Google Scholar 

  44. Rocco G, Internullo E, Cassivi SD, Van Raemdonck D, Ferguson MK (2008) The variability of practice in minimally invasive thoracic surgery for pulmonary resections. Thorac Surg Clin 18:235–247. doi:10.1016/j.thorsurg.2008.06.002

    Article  PubMed  Google Scholar 

  45. Petersen RH, Hansen HJ (2012) Learning curve associated with VATS lobectomy. Ann Cardiothorac Surg 1:47–50. doi:10.3978/478

    PubMed  PubMed Central  Google Scholar 

  46. Li X, Wang J, Ferguson MK (2014) Competence versus mastery: the time course for developing proficiency in video-assisted thoracoscopic lobectomy. J Thorac Cardiovasc Surg 147:1150–1154. doi:10.1016/j.jtcvs.2013.11.036

    Article  PubMed  Google Scholar 

  47. Ferguson J, Walker W (2006) Developing a VATS lobectomy programme—Can VATS lobectomy be taught? Eur J Cardiothorac Surg 29:806–809. doi:10.1016/j.ejcts.2006.02.012

    Article  PubMed  Google Scholar 

  48. McKenna RJ (2008) Complications and learning curves for video-assisted thoracic surgery lobectomy. Thorac Surg Clin 18:275–280. doi:10.1016/j.thorsurg.2008.04.004

    Article  PubMed  Google Scholar 

  49. Gjeraa K, Spanager L, Konge L, Petersen RH, Ostergaard D (2016) Non-technical skills in minimally invasive surgery teams: a systematic review. Surg Endosc. doi:10.1007/s00464-016-4890-1

    PubMed  Google Scholar 

  50. Savran MM, Hansen HJ, Petersen RH, Walker W, Schmid T, Bojsen SR, Konge L (2015) Development and validation of a theoretical test of proficiency for video-assisted thoracoscopic surgery (VATS) lobectomy. Surg Endosc 29:2598–2604. doi:10.1007/s00464-014-3975-y

    Article  PubMed  Google Scholar 

  51. Vilmann P, Frost Clementsen P, Colella S, Siemsen M, De Leyn P, Dumonceau JM, Herth FJ, Larghi A, Vazquez-Sequeiros E, Hassan C, Crombag L, Korevaar DA, Konge L, Annema JT (2015) Combined endobronchial and esophageal endosonography for the diagnosis and staging of lung cancer: European Society of gastrointestinal endoscopy (ESGE) guideline, in cooperation with the European Respiratory Society (ERS) and the European Society of thor. Eur J Cardiothorac Surg 48:1–15. doi:10.1093/ejcts/ezv194

    Article  PubMed  Google Scholar 

  52. Aggarwal R, Mytton OT, Derbrew M, Hananel D, Heydenburg M, Issenberg B, MacAulay C, Mancini ME, Morimoto T, Soper N, Ziv A, Reznick R (2010) Training and simulation for patient safety. Qual Saf Health Care 19(Suppl 2):i34–i43. doi:10.1136/qshc.2009.038562

    Article  PubMed  Google Scholar 

  53. Barsuk JH, Cohen ER, Wayne DB, Siddall VJ, McGaghie WC (2016) Developing a simulation-based mastery learning curriculum. Simul Healthc J Soc Simul Healthc 11:52–59. doi:10.1097/SIH.0000000000000120

    Article  Google Scholar 

  54. Mcgaghie WC (2015) Mastery learning : it is time for medical education to join the 21st century. Acad Med 90:1–4. doi:10.1097/ACM.0000000000000911

    Article  Google Scholar 

  55. Konge L, Vilmann P, Clementsen P, Annema JT, Ringsted C (2012) Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer. Endoscopy 44:928–933. doi:10.1055/s-0032-1309892

    Article  CAS  PubMed  Google Scholar 

  56. Tong BC, Gustafson MR, Balderson SS, D’Amico TA, Meyerson SL (2012) Validation of a thoracoscopic lobectomy simulator. Eur J Cardiothorac Surg 42:364–369. doi:10.1093/ejcts/ezs012

    Article  PubMed  Google Scholar 

  57. Meyerson SL, Tong BC, Balderson SS, D’Amico TA, Phillips JD, Decamp MM, Darosa DA (2012) Needs assessment for an errors-based curriculum on thoracoscopic lobectomy. Ann Thorac Surg 94:368–373. doi:10.1016/j.athoracsur.2012.04.023

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

We would like to thank medical student Julia Dagnæs for her invaluable help with managing the simulators during the period of data collection.

Author information

Authors and Affiliations

  1. Department of Cardiothoracic Surgery, Sect. 2152, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark

    Katrine Jensen, Henrik Jessen Hansen, René Horsleben Petersen & Jesper Holst Pedersen

  2. Copenhagen Academy for Medical Education and Simulation (CAMES), Section 5404, University of Copenhagen and Capital Region, Rigshospitalet, Blegdamsvej 9, 2100 København Ø, Copenhagen, Denmark

    Katrine Jensen & Lars Konge

  3. JMC Simulation Unit, The Juliane Marie Centre, Section 4704, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark

    Flemming Bjerrum

Authors
  1. Katrine Jensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Flemming Bjerrum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Henrik Jessen Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. René Horsleben Petersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jesper Holst Pedersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lars Konge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Katrine Jensen.

Ethics declarations

Disclosures

Katrine Jensen’s Ph.D.project is partly funded by a grant from The Danish Cancer Society (Kræftens Bekæmpelse, “Knæk Cancer”). Henrik Jessen Hansen and René Horsleben Petersen are at the Speakers Bureau of Covidien. Flemming Bjerrum, Jesper Holst Pedersen and Lars Konge have no conflicts of interest or financial ties to disclose.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jensen, K., Bjerrum, F., Hansen, H.J. et al. Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy. Surg Endosc 31, 2520–2528 (2017). https://doi.org/10.1007/s00464-016-5254-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-016-5254-6

Keywords

Search

Navigation