Skip to main content
SpringerLink
Log in

Assessing VATS competence based on simulated lobectomies of all five lung lobes

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Objectives

To determine the number of procedures and expert raters necessary to provide a reliable assessment of competence in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy.

Methods

Three randomly selected VATS lobectomies were performed on a virtual reality simulator by participants with varying experience in VATS. Video recordings of the procedures were independently rated by three blinded VATS experts using a modified VATS lobectomy assessment tool (VATSAT). The unitary framework of validity was used to describe validity evidence, and generalizability theory was used to explore the reliability of different assessment options.

Results

Forty-one participants (22 novices, 10 intermediates, and 9 experienced) performed a total of 123 lobectomies. Internal consistency reliability, inter-rater reliability, and test–retest reliability were 0.94, 0.85, and 0.90, respectively. Generalizability theory found that a minimum of two procedures and four raters or three procedures and three raters were needed to ensure the overall reliability of 0.8. ANOVA showed significant differences in test scores between the three groups (P < 0.001). A pass/fail level of 19 out of 25 points was established using the contrasting groups’ standard setting method, leaving one false positive (one novice passed) and zero false negatives (all experienced passed).

Conclusion

We demonstrated validity evidence for a VR simulator test with different lung lobes, and a credible pass/fail level was identified. Our results can be used to implement a standardized mastery learning training program for trainees in VATS lobectomies that ensures that everyone reaches basic competency before performing supervised operations on patients.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

VATS:

Video-Assisted Thoracoscopic Surgery

VATSAT:

Video-Assisted Thoracoscopic Assessment Tool

VR:

Virtual Reality

CAMES:

Copenhagen Academy for Medical Education and Simulation

REDCap:

Research Electronic Data Capture

References

  1. https://www.wcrf.org/dietandcancer/cancer-trends/lung-cancer-statistics

  2. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A (2018) Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 68(6):394–424

    Article  PubMed  Google Scholar 

  3. Rosen JE, Keshava HB, Yao X, Kim AW, Detterbeck FC, Boffa DJ (2016) The natural history of operable non-small cell lung cancer in the national cancer database. Ann Thorac Surg 101(5):1850–1855

    Article  PubMed  Google Scholar 

  4. Howington JA, Blum MG, Chang AC, Balekian AA, Murthy SC (2013) Treatment of stage I and II non-small cell lung cancer: diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 143(5 Suppl):e278S-e313S

    Article  CAS  PubMed  Google Scholar 

  5. Bendixen M, Jørgensen OD, Kronborg C, Andersen C, Licht PB (2016) Postoperative pain and quality of life after lobectomy via video-assisted thoracoscopic surgery or anterolateral thoracotomy for early stage lung cancer: a randomized controlled trial. Lancet Oncol 17(6):836–844

    Article  PubMed  Google Scholar 

  6. Paul S, Altorki NK, Sheng S, Lee PC, Harpole DH, Onaitis MW et al (2010) Thoracoscopic lobectomy is associated with lower morbidity than open lobectomy: a propensity-matched analysis from the STS database. J Thorac Cardiovasc Surg 139(2):366–378

    Article  PubMed  Google Scholar 

  7. Swanson SJ, Meyers BF, Gunnarsson CL, Moore M, Howington JA, Maddaus MA et al (2012) Video-assisted thoracoscopic lobectomy is less costly and morbid than open lobectomy: a retrospective multiinstitutional database analysis. Ann Thorac Surg 93(4):1027–1032

    Article  PubMed  Google Scholar 

  8. Kaseda S, Aoki T, Hangai N, Shimizu K (2000) Better pulmonary function and prognosis with video-assisted thoracic surgery than with thoracotomy. Ann Thorac Surg 70(5):1644–1646

    Article  CAS  PubMed  Google Scholar 

  9. Decaluwe H, Petersen RH, Hansen HJ, Piwkowski C, Augustin F, Brunelli A et al (2015) Major intraoperative complications during video-assisted thoracoscopic anatomical lung resections: an intention-to-treat analysis. Eur J Cardiothorac Surg 48(4):588–599

    Article  PubMed  Google Scholar 

  10. Konge L, Petersen RH, Ringsted C (2018) Developing competency in video-assisted thoracic surgery (VATS) lobectomy. J Thorac Dis 10(Suppl 17):2025–2028

    Article  Google Scholar 

  11. Jensen K, Bjerrum F, Hansen HJ, Petersen RH, Pedersen JH, Konge L (2017) Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy. Surg Endosc 31(6):2520–2528

    Article  PubMed  Google Scholar 

  12. Nashaat A, Sidhu HS, Yatham S, Al-Azzawi M, Preece R (2019) Simulation training for lobectomy: a review of current literature and future directions†. Eur J Cardiothorac Surg. 55(3):386–394

    Article  PubMed  Google Scholar 

  13. Bonrath EM, Dedy NJ, Gordon LE, Grantcharov TP (2015) Comprehensive surgical coaching enhances surgical skill in the operating room: a randomized controlled trial. Ann Surg 262(2):205–212

    Article  PubMed  Google Scholar 

  14. Lodge D, Grantcharov T (2011) Training and assessment of technical skills and competency in cardiac surgery. Eur J Cardiothorac Surg 39(3):287–293

    Article  PubMed  Google Scholar 

  15. Brunelli A, Falcoz PE, D’Amico T, Hansen H, Lim E, Massard G et al (2014) European guidelines on structure and qualification of general thoracic surgery. Eur J Cardiothorac Surg 45(5):779–786

    Article  PubMed  Google Scholar 

  16. Jensen K, Bjerrum F, Hansen HJ, Petersen RH, Pedersen JH, Konge L (2017) Using virtual reality simulation to assess competence in video-assisted thoracoscopic surgery (VATS) lobectomy. Surg Endosc 31(6):2520–2528

    Article  PubMed  Google Scholar 

  17. Fisher RA, Dasgupta P, Mottrie A, Volpe A, Khan MS, Challacombe B, Ahmed K (2015) An over-view of robot assisted surgery curricula and the status of their validation. Int J Surg 13:115–123

    Article  PubMed  Google Scholar 

  18. Haidari TA, Bjerrum F, Hansen HJ, Konge L, Petersen RH (2022) Simulation-based VATS resection of the five lung lobes: a technical skills test. Surg Endosc. 36(2):1234–1242

    Article  PubMed  Google Scholar 

  19. Epstein RM, Cassel CK, Epstein RM, de Galan BE, van Gurp PJ, Stuyt PM (2007) Assessment in medical education. N Engl J Med 356:387–396

    Article  CAS  PubMed  Google Scholar 

  20. Konge L, Larsen KR, Clementsen P, Arendrup H, von Buchwald C, Ringsted C (2012) Reliable and valid assessment of clinical bronchoscopy performance. Respiration 83(1):53–60

    Article  PubMed  Google Scholar 

  21. Aggarwal R, Grantcharov T, Moorthy K, Milland T, Darzi A (2008) Toward feasible, valid, and reliable video-based assessments of technical surgical skills in the operating room. Ann Surg 247(2):372–379

    Article  PubMed  Google Scholar 

  22. Carlsen CG, Lindorff-Larsen K, Funch-Jensen P, Lund L, Charles P, Konge L (2014) Reliable and valid assessment of Lichtenstein hernia repair skills. Hernia 18(4):543–548

    Article  CAS  PubMed  Google Scholar 

  23. Kibble JD (2017) Best practices in summative assessment. Adv Physiol Educ. 41(1):110–119

    Article  PubMed  Google Scholar 

  24. Petersen RH, Gjeraa K, Jensen K, Møller LB, Hansen HJ, Konge L (2018) Assessment of competence in video-assisted thoracoscopic surgery lobectomy: a Danish nationwide study. J Thorac Cardiovasc Surg 156(4):1717–1722

    Article  PubMed  Google Scholar 

  25. Jensen K, Hansen HJ, Petersen RH, Neckelmann K, Vad H, Møller LB et al (2019) Evaluating competency in video-assisted thoracoscopic surgery (VATS) lobectomy performance using a novel assessment tool and virtual reality simulation. Surg Endosc 33(5):1465–1473

    Article  PubMed  Google Scholar 

  26. Cook DA, Brydges R, Zendejas B, Hamstra SJ, Hatala R (2013) Mastery learning for health professionals using technology-enhanced simulation: a systematic review and meta-analysis. Acad Med 88(8):1178–1186

    Article  PubMed  Google Scholar 

  27. Hansen HJ, Petersen RH, Christensen M (2011) Video-assisted thoracoscopic surgery (VATS) lobectomy using a standardized anterior approach. Surg Endosc 25:1263–1269

    Article  PubMed  Google Scholar 

  28. Messick S (1898) Meaning and values in test validation: the science and ethics of assessment. Educ Res 18(2):5–11

    Article  Google Scholar 

  29. Jørgensen M, Konge L, Subhi Y (2018) Contrasting groups’ standard setting for consequences analysis in validity studies: reporting considerations. Adv Simul 9(3):5

    Article  Google Scholar 

  30. Pugh CM, Hashimoto DA, Korndorffer JR Jr (2021) The what? How? And Who?Of video-based assessment. Am J Surg 221(1):13–18

    Article  PubMed  Google Scholar 

  31. Downing SM (2004) Reliability: on the reproducibility of assessment data. Med Educ 38:1006–1012

    Article  PubMed  Google Scholar 

  32. Andersen SAW, Nayahangan LJ, Park YS, Konge L (2021) Use of generalizability theory for exploring reliability of and sources of variance in assessment of technical skills: a systematic review and meta-analysis. Acad Med. 96(11):1609–1619

    Article  PubMed  Google Scholar 

  33. Van de Ridder JM, Stokking KM, McGaghie WC, ten Cate OT (2008) What is feedback in clinical education? Med Educ 42(2):189–197

    Article  PubMed  Google Scholar 

  34. Vilmann AS, Norsk D, Svendsen MBS, Reinhold R, Svendsen LB, Park YS, Konge L (2018) Computerized feedback during colonoscopy training leads to improved performance: a randomized trial. Gastrointest Endosc 88(5):869–876

    Article  PubMed  Google Scholar 

  35. Ostberg NP, Zafar MA, Elefteriades JA (2021) Machine learning: principles and applications for thoracic surgery. Eur J Cardiothorac Surg. 60(2):213–221

    Article  PubMed  Google Scholar 

  36. Borgersen NJ, Naur TMH, Sørensen SMD, Bjerrum F, Konge L, Subhi Y, Thomsen ASS (2018) Gathering validity evidence for surgical simulation: a systematic review. Ann Surg 267(6):1063–1068

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

None.

Funding

Research fund at the Heart Centre, Rigshospitalet, Copenhagen, Denmark.

Author information

Authors and Affiliations

  1. Department of Cardiothoracic Surgery, University Hospital of Copenhagen, Inge Lehmanns Vej 7, Section 2152, Rigshospitalet, Copenhagen, Denmark

    Tamim Ahmad Haidari, Henrik Vad, Henrik Jessen Hansen & René Horsleben Petersen

  2. Department of Surgery, Zealand University Hospital, Køge, Copenhagen, Denmark

    Flemming Bjerrum

  3. Copenhagen Academy for Medical Education and Simulation, Center for Human Resources and Education, Rigshospitalet, Copenhagen, Denmark

    Flemming Bjerrum & Lars Konge

  4. Department of Cardiothoracic and Vascular Surgery & Department of Clinical Medicine, Aarhus University Hospital, Aarhus, Central Denmark Region, Denmark

    Thomas Decker Christensen

  5. Department of Cardiothoracic Surgery, Aalborg University Hospital, Aalborg, North Jutland, Denmark

    Lars Borgbjerg Møller

  6. Department of Clinical Medicine, University of Copenhagen, Rigshospitalet, Copenhagen, Denmark

    Tamim Ahmad Haidari, Lars Konge & René Horsleben Petersen

Authors
  1. Tamim Ahmad Haidari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Flemming Bjerrum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Decker Christensen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Henrik Vad
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lars Borgbjerg Møller
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Henrik Jessen Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lars Konge
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. René Horsleben Petersen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tamim Ahmad Haidari.

Ethics declarations

Disclosures

FB, LK, HV and LBM have no conflicts of interests to disclose. RHP: Speaker fee Medtronic, AMBU, AstraZeneca and Advisory Board: AstraZeneca, Roche and MSD. HJH: Speaker fee Medtronic, Medela and BD/Bard. TDC has been on the speaker bureaus for AstraZeneca, Boehringer-Ingelheim, Pfizer, Roche Diagnostics, Takeda, Merck Sharp & Dohme (MSD) and Bristol-Myers Squibb and has been in an Advisory Board for Bayer and Merck Sharp & Dohme (MSD).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

Modified video-assisted thoracoscopic surgery assessment tool (25).

 

1

2

3

4

5

1. Dissection of the hilum and veins

Dissection is unsafe and the trainee cannot remove connective tissue from the hilum to identify the veins and prepare them for stapling. Dissection is finally done by supervisor

 

Connective tissue and lymph nodes if necessary are removed from the hilum and the veins are identified and prepared for stapling with some hands-on guidance from supervisor.

 

The hilum is properly and safely dissected and the veins are identified and prepared for stapling without help from supervisor. Trainee checks for single pulmonary vein before stapling

2. Dissection of the arteries

Dissection is unsafe and the trainee cannot identify the arteries to the affected lobe and prepare them for stapling, Dissection is finally done by supervisor.

 

The pulmonary artery and arteries for the affected lobe are identified, connective tissue and lymph nodes if necessary are removed and the arteries are prepared for stapling with some hands-on guidance from supervisor.

 

The pulmonary artery and arteries for the affected lobe are identified, proper and safely dissected and prepared for stapling without help from supervisor. Trainee checks for aria-tomical variations of the arteries before stapling.

3. Dissection of the bronchus

Dissection is unsafe and the trainee cannot identify the bronchus and prepare it for stapling. Dissection is finally done by supervisor.

 

The bronchus and bronchial arteries to the affected lobe are identified, connective tissue and lymph nodes if necessary are removed and the bronchus is prepared for stapling with some hands-on guidance from supervisor.

 

The bronchus and bronchial arteries are proper and safely dissected and prepared for stapling without help from supervisor.

4. Respect for tissue and structures

The trainee uses diathermy! instruments to close to vital structures and tissue (nerves, oesophagus, vessels, lung parenchyma in affected/ adjacent lobes) and causes unacceptable inadvertent damage.

 

The trainee gently manipu- lates tissue and vital strut- tures (nerves, oesophagus, vessels, lung parenchyma in affected/adjacent lobes) but occasionally causes inadvertent damage.

 

The trainee consistently demonstrates appropriate handling of tissue and vital structures with minimal inadvertent damage.,

5. Technical skills in general

The trainee handles instruments incorrectly and with too much force, does not keep instruments in the field of vision, is not familiar with most instruments and lacks fluidity and accuracy of hand movements.

 

The trainee handles instruments adequately, keeps instruments in the field of vision most of the time, is familiar with most instruments but is occasionally stiff and awkward.

 

The trainee demonstrates complete familiarity with all instruments and handles these correctly and not with too much force, keeps instruments in the field of vision all of the time and has excellent fluidity and accuracy of hand movements.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Haidari, T.A., Bjerrum, F., Christensen, T.D. et al. Assessing VATS competence based on simulated lobectomies of all five lung lobes. Surg Endosc 36, 8067–8075 (2022). https://doi.org/10.1007/s00464-022-09235-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-022-09235-5

Keywords

Search

Navigation