Skip to main content

Advertisement

SpringerLink
Log in

Laparoscopic cholecystectomy critical view of safety (LC-CVS): a multi-national validation study of an objective, procedure-specific assessment using video-based assessment (VBA)

  • 2023 SAGES Oral
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

A novel 6-item objective, procedure-specific assessment for laparoscopic cholecystectomy incorporating the critical view of safety (LC-CVS OPSA) was developed to support trainee formative and summative assessments. The LC-CVS OPSA included two retraction items (fundus and infundibulum retraction) and four CVS items (hepatocystic triangle visualization, gallbladder-liver separation, cystic artery identification, and cystic duct identification). The scoring rubric for retraction consisted of poor (frequently outside of defined range), adequate (minimally outside of defined range) and excellent (consistently inside defined range) and for CVS items were “poor—unsafe”, “adequate—safe”, or “excellent—safe”.

Methods

A multi-national consortium of 12 expert LC surgeons applied the OPSA—LC CVS to 35 unique LC videos and one duplicate video. Primary outcome measure was inter-rater reliability as measured by Gwet’s AC2, a weighted measure that adjusts for scales with high probability of random agreement. Analysis of the inter-rater reliability was conducted on a collapsed dichotomous scoring rubric of “poor—unsafe” vs. “adequate/excellent—safe”.

Results

Inter-rater reliability was high for all six items ranging from 0.76 (hepatocystic triangle visualization) to 0.86 (cystic duct identification). Intra-rater reliability for the single duplicate video was substantially higher across the six items ranging from 0.91 to 1.00.

Conclusions

The novel 6-item OPSA LC CVS demonstrated high inter-rater reliability when tested with a multi-national consortium of LC expert surgeons. This brief instrument focused on safe surgical practice was designed to support the implementation of entrustable professional activities into busy surgical training programs. Instrument use coupled with video-based assessments creates novel datasets with the potential for artificial intelligence development including computer vision to drive assessment automation.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Lindeman B, Petrusa E, Phitayakorn R (2017) Entrustable professional activities (EPAs) and applications to surgical training. Am Coll Surg. https://www.facs.org/for-medical-professionals/news-publications/journals/rise/articles/entrustable/. Accessed 1 Mar 2023

  2. Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, Harris P, Glasgow NJ, Campbell C, Dath D, Harden RM, Iobst W, Long DM, Mungroo R, Richardson DL, Sherbino J, Silver I, Taber S, Talbot M, Harris KA (2010) Competency-based medical education: theory to practice. Med Teach. https://doi.org/10.3109/0142159X.2010.501190

    Article  PubMed  Google Scholar 

  3. Singh P, Aggarwal R, Zevin B, Grantcharov T, Darzi A (2014) A global Delphi consensus study on defining and measuring quality in surgical training. J Am Coll Surg. https://doi.org/10.1016/j.jamcollsurg.2014.03.051

    Article  PubMed  Google Scholar 

  4. Huang RJ, Limsui D, Triadafilopoulos G (2018) Video-based performance assessment in endoscopy: moving beyond “see one, do one, teach one”. Gastrointest Endosc. https://doi.org/10.1016/j.gie.2017.09.014

    Article  PubMed  PubMed Central  Google Scholar 

  5. Ibrahim AM, Varban OA, Dimick JB (2016) Novel uses of video to accelerate the surgical learning curve. J Laparoendosc Adv Surg Tech A. https://doi.org/10.1089/lap.2016.0100

    Article  PubMed  PubMed Central  Google Scholar 

  6. Ducournau F, Meyer N, Xavier F, Facca S, Liverneaux P (2021) Learning a MIPO technique for distal radius fractures: mentoring versus simple experience versus deliberate practice. Orthop Traumatol Surg Res. https://doi.org/10.1016/j.otsr.2021.102939

    Article  PubMed  Google Scholar 

  7. Missori G, Serra F, Gelmini R (2022) A narrative review about difficult laparoscopic cholecystectomy: technical tips. Laparosc Surg. https://doi.org/10.21037/ls-22-16

    Article  Google Scholar 

  8. Strasberg SM (2017) A perspective on the critical view of safety in laparoscopic cholecystectomy. Ann Laparosc Endosc Surg. https://doi.org/10.21037/ales.2017.04.08

    Article  Google Scholar 

  9. Kapoor VK (2016) Medico-legal aspects of bile duct injury. J Minim Access Surg. https://doi.org/10.4103/0972-9941.169995

    Article  PubMed  PubMed Central  Google Scholar 

  10. Flum DR, Cheadle A, Prela C, Dellinger EP, Chan L (2003) Bile duct injury during cholecystectomy and survival in Medicare beneficiaries. JAMA. https://doi.org/10.1001/jama.290.16.2168

    Article  PubMed  Google Scholar 

  11. Bektas H, Schrem H, Winny M, Klempnauer J (2007) Surgical treatment and outcome of iatrogenic bile duct lesions after cholecystectomy and the impact of different clinical classification systems. Br J Surg. https://doi.org/10.1002/bjs.5752

    Article  PubMed  Google Scholar 

  12. Brunt LM, Deziel DJ, Telem DA, Strasberg SM, Aggarwal R, Asbun H, Bonjer J, McDonald M, Alseidi A, Ujiki M, Riall TS, Hammill C, Moulton C-A, Pucher PH, Parks RW, Ansari MT, Connor S, Dirks RC, Anderson B, Altieri MS, Tsamalaidze L, Stefanidis D (2020) Safe cholecystectomy multi-society practice guideline and state of the art consensus conference on prevention of bile duct injury during cholecystectomy. Ann Surg. https://doi.org/10.1097/sla.0000000000003791

    Article  PubMed  Google Scholar 

  13. Wakabayashi G, Iwashita Y, Hibi T, Takada T, Strasberg SM, Asbun HJ, Endo I, Umezawa A, Asai K, Suzuki K, Mori Y, Okamoto K, Pitt HA, Han H-S, Hwang T-L, Yoon Y-S, Yoon D-S, Choi I-S, Huang WS-W, Giménez ME, Garden OJ, Gouma DJ, Belli G, Dervenis C, Jagannath P, Chan AC, Lau WY, Liu K-H, Su C-H, Misawa T, Nakamura M, Horiguchi A, Tagaya N, Fujioka S, Higuchi R, Shikata S, Noguchi Y, Ukai T, Yokoe M, Cherqui D, Honda G, Sugioka A, de Santibañes E, Supe AN, Tokumura H, Kimura T, Yoshida M, Mayumi T, Kitano S, Inomata M, Hirata K, Sumiyama Y, Inui K, Yamamoto M (2018) Tokyo guidelines 2018: surgical management of acute cholecystitis: safe steps in laparoscopic cholecystectomy for acute cholecystitis (with videos). J Hepatobiliary Pancreat Sci. https://doi.org/10.1002/jhbp.517

    Article  PubMed  Google Scholar 

  14. Eikermann M, Siegel R, Broeders I, Dziri C, Fingerhut A, Gutt C, Jaschinski T, Nassar A, Paganini AM, Pieper D, Targarona E, Schrewe M, Shamiyeh A, Strik M, Neugebauer EA (2012) Prevention and treatment of bile duct injuries during laparoscopic cholecystectomy: the clinical practice guidelines of the European Association for Endoscopic Surgery (EAES). Surg Endosc. https://doi.org/10.1007/s00464-012-2511-1

    Article  PubMed  Google Scholar 

  15. Jin Y, Liu R, Chen Y, Liu J, Zhao Y, Wei A, Li Y, Li H, Xu J, Wang X, Li A (2022) Critical view of safety in laparoscopic cholecystectomy: a prospective investigation from both cognitive and executive aspects. Front Surg. https://doi.org/10.3389/fsurg.2022.946917

    Article  PubMed  PubMed Central  Google Scholar 

  16. Deal SB, Alseidi AA (2017) Concerns of quality and safety in public domain surgical education videos: an assessment of the critical view of safety in frequently used laparoscopic cholecystectomy videos. J Am Coll Surg. https://doi.org/10.1016/j.jamcollsurg.2017.08.01

    Article  PubMed  Google Scholar 

  17. Nijssen MAJ, Schreinemakers JMJ, van der Schelling GP, Crolla RMPH, Rijken AM (2016) Improving critical view of safety in laparoscopic cholecystectomy by teaching interventions. J Surg Educ. https://doi.org/10.1016/j.jsurg.2015.11.015

    Article  PubMed  Google Scholar 

  18. Jung YK, Kwon YJ, Choi D, Lee KG (2016) What is the safe training to educate the laparoscopic cholecystectomy for surgical residents in early learning curve. J Minim Invasive Surg. https://doi.org/10.7602/jmis.2015.19.2.70

    Article  Google Scholar 

  19. Nickel F, Hendrie JD, Stock C, Salama M, Preukschas AA, Senft JD, Kowalewski KF, Wagner M, Kenngott HG, Linke GR, Fischer L, Müller-Stich BP (2016) Direct observation versus endoscopic video recording-based rating with the objective structured assessment of technical skills for training of laparoscopic cholecystectomy. Eur Surg Res. https://doi.org/10.1159/000444449

    Article  PubMed  Google Scholar 

  20. Kramp KH, van Det MJ, Hoff C, Lamme B, Veeger NJGM, Pierie J-PEN (2015) Validity and reliability of global operative assessment of laparoscopic skills (goals) in novice trainees performing a laparoscopic cholecystectomy. J Surg Educ. https://doi.org/10.1016/j.jsurg.2014.08.006

    Article  PubMed  Google Scholar 

  21. Hatala R, Cook DA, Brydges R, Hawkins R (2015) Constructing a validity argument for the Objective Structured Assessment of Technical Skills (OSATS): a systematic review of validity evidence. Adv Health Sci Educ Theory Pract. https://doi.org/10.1007/s10459-015-9593-1

    Article  PubMed  Google Scholar 

  22. Kramp KH, van Det MJ, Veeger NJGM, Pieria J-PEN (2015) Validity, reliability and support for implementation of independence-scaled procedural assessment in laparoscopic surgery. Surg Endosc. https://doi.org/10.1007/s00464-015-4254-2

    Article  PubMed  PubMed Central  Google Scholar 

  23. Glarner CE, McDonald RJ, Smith AB, Leverson GE, Peyre S, Pugh CM, Greenberg CC, Greenberg JA, Foley EF (2013) Utilizing a novel tool for the comprehensive assessment of resident operative performance. J Surg Educ. https://doi.org/10.1016/j.jsurg.2013.07.009

    Article  PubMed  Google Scholar 

  24. Adrales G, Ardito F, Chowbey P, Morales-Conde S, Ferreres AR, Hensman C, Martin D, Matthaei H, Ramshaw B, Roberts JK, Schrem H, Sharma A, Tabiri S, Vibert E, Woods MS (2023) A multi-national, video-based qualitative study to refine training guidelines for assigning an “unsafe” score in laparoscopic cholecystectomy critical view of safety (in press)

  25. Caresyntax (2023) Platform. https://caresyntax.com/platform/. Accessed 1 Mar 2023

  26. Gwet KL (2008) Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. https://doi.org/10.1348/000711006x126600

    Article  PubMed  Google Scholar 

  27. Gwet KL (2014) Handbook of inter-rater reliability: the definitive guide to measuring the extent of agreement among raters, 4th edn. Advanced Analytics LLC, Gaithersburg

    Google Scholar 

  28. Wongpakaran N, Wongpakaran T, Wedding D, Gwet KL (2013) A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol. https://doi.org/10.1186/1471-2288-13-61

    Article  PubMed  PubMed Central  Google Scholar 

  29. Blood E, Spratt KF (2007) Disagreement on agreement: two alternative agreement coefficients, SAS Global Forum 2007. https://support.sas.com/resources/papers/proceedings/proceedings/forum2007/186-2007.pdf. Accessed 1 Mar 2023

  30. Chevallay M, Liot E, Fournier I, Abbassi Z, Peloso A, Hagen ME, Mönig SP, Morel P, Toso C, Buchs N, Miskovic D, Ris F, Jung MK (2022) Implementation and validation of a competency assessment tool for laparoscopic cholecystectomy. Surg Endosc. https://doi.org/10.1007/s00464-022-09264-0

    Article  PubMed  PubMed Central  Google Scholar 

  31. Van Zwieten TH, Okkema S, Kramp KH, de Jong K, Van Det MJ, Pierie J-PEN (2022) Procedure-based assessment for laparoscopic cholecystectomy can replace global rating scales. Minim Invasiv Ther Allied Technol. https://doi.org/10.1080/13645706.2021.1995000

    Article  Google Scholar 

  32. Bethlehem MS, Kramp KH, van Det MJ, ten Cate Hoedemaker HO, Veeger NJ, Pierie JP (2014) Development of a standardized training course for laparoscopic procedures using Delphi methodology. J Surg Educ. https://doi.org/10.1016/j.jsurg.2014.04.009

    Article  PubMed  Google Scholar 

  33. Kramp KH, van Det MJ, Veeger NJGM, Pierie J-PEN (2015) Validity, reliability and support for implementation of independence-scaled procedural assessment in laparoscopic surgery. Surg Endosc. https://doi.org/10.1007/s00464-015-4254-2

    Article  PubMed  PubMed Central  Google Scholar 

  34. Champagne BJ (2013) Effective teaching and feedback strategies in the OR and beyond. Clin Colon Rectal Surg. https://doi.org/10.1055/s-0033-1356725

    Article  PubMed  PubMed Central  Google Scholar 

  35. American Board of Surgery (2022) Entrustable professional activities. https://www.absurgery.org/default.jsp?epahome. Accessed 1 Mar 2023

  36. Stahl CC, Collins E, Jung SA, Rosser AA, Kraut AS, Schnapp BH, Westergaard M, Hamedani AG, Minter RM, Greenberg JA (2020) Implementation of entrustable professional activities into a general surgery residency. J Surg Educ. https://doi.org/10.1016/j.jsurg.2020.01.012

    Article  PubMed  PubMed Central  Google Scholar 

  37. Brasel KJ, Klingensmith ME, Englander R, Grambau M, Buyske J, Sarosi G, Minter R (2019) Entrustable professional activities in general surgery: development and implementation. J Surg Educ. https://doi.org/10.1016/j.jsurg.2019.04.003

    Article  PubMed  Google Scholar 

  38. Vedula SS, Ishii M, Hager GD (2017) Objective assessment of surgical technical skill and competency in the operating room. Annu Rev Biomed Eng. https://doi.org/10.1146/annurev-bioeng-071516-044435

    Article  PubMed  PubMed Central  Google Scholar 

  39. Egert M, Steward JE, Sundaram CP (2020) Machine learning and artificial intelligence in surgical fields. Indian J Surg Oncol. https://doi.org/10.1007/s13193-020-01166-8

    Article  PubMed  PubMed Central  Google Scholar 

  40. Hashimoto DA, Rosman G, Witkowski ER, Stafford C, Navarette-Welton AJ, Rattner DW, Lillemoe KD, Rus DL, Meireles OR (2019) Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg. https://doi.org/10.1097/sla.0000000000003460

    Article  PubMed  Google Scholar 

  41. Mascagni P, Alapatt D, Urade T, Vardazaryan A, Mutter D, Marescaux J, Costamagna G, Dallemagne B, Padoy N (2021) A computer vision platform to automatically locate critical events in surgical videos: documenting safety in laparoscopic cholecystectomy. Ann Surg. https://doi.org/10.1097/SLA.0000000000004736

    Article  PubMed  Google Scholar 

  42. Vedula SS, Hager GD (2017) Surgical data science: the new knowledge domain. Innov Surg Sci. https://doi.org/10.1515/iss-2017-0004

    Article  PubMed  PubMed Central  Google Scholar 

  43. Watanabe Y, Bilgic E, Lebedeva E, McKendy KM, Feldman LS, Fried GM, Vassiliou MC (2016) A systematic review of performance assessment tools for laparoscopic cholecystectomy. Surg Endosc. https://doi.org/10.1007/s00464-015-4285-8

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors wish to thank Justin Smith for analytical support and Health Analytics, LLC for administrative and medical writing support in preparing the manuscript.

Funding

This work was supported by Caresyntax sponsored this project.

Author information

Authors and Affiliations

  1. Johns Hopkins University School of Medicine, 600 N. Wolfe St., Blalock 618, Baltimore, MD, 21287, USA

    Gina Adrales

  2. Hepatobiliary Surgery Unit, Fondazione Policlinico Universitario Agostino Gemelli, IRCCS, Catholic University, Rome, Italy

    Francesco Ardito

  3. Institute of Laparoscopic, Endoscopic and Bariatric Surgery, Max Super Specialty Hospital, Saket, New Delhi, India

    Pradeep Chowbey & Anil Sharma

  4. Unit of Innovation in Minimally Invasive Surgery, University Hospital Virgen del Rocío, University of Sevilla, Sevilla, Spain

    Salvador Morales-Conde

  5. Department of Surgery, University of Buenos Aires, Buenos Aires, Argentina

    Alberto R. Ferreres

  6. Department of Surgery & LapSurgery, Monash University, Melbourne, Australia

    Chrys Hensman

  7. Division of Critical Care/Acute Care Surgery, University of Minnesota, Minneapolis, MN, USA

    David Martin

  8. Department of Surgery, University Medical Center, Bonn, Germany

    Hanno Matthaei

  9. CQInsights PBC, Knoxville, TN, USA

    Bruce Ramshaw

  10. Liver Transplant and HPB Surgery, University Hospitals Birmingham NHS Trust, Birmingham, UK

    J. Keith Roberts

  11. General, Visceral and Transplant Surgery, Medical University Graz, Graz, Austria

    Harald Schrem

  12. University for Development Studies-School of Medicine and Health Sciences, Tamale Teaching Hospital, Tamales, Ghana

    Stephen Tabiri

  13. Centre Hépato-Biliaire, Paul Brousse Hospital, AP-HP, Villejuif, France

    Eric Vibert

  14. Caresyntax Corporation, Boston, MA, USA

    Bruce Ramshaw & Michael S. Woods

Authors
  1. Gina Adrales
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesco Ardito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pradeep Chowbey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Salvador Morales-Conde
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alberto R. Ferreres
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chrys Hensman
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. David Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hanno Matthaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Bruce Ramshaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. J. Keith Roberts
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Harald Schrem
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Anil Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Stephen Tabiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Eric Vibert
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Michael S. Woods
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gina Adrales.

Ethics declarations

Disclosures

Adrales reported receipt of support for this manuscript and consulting fees from Caresyntax. Ardito reported receipt of support for this manuscript, consulting fees, payment or honoraria, participation on a data safety monitoring or advisory board and a leadership or fiduciary role on a review board with Caresyntax. Chowbey and Matthaei reported receipt of support for this manuscript from Caresyntax. Morales-Conde, Ferreres, and Martin reported receipt of support for this manuscript and payment or honoraria from Caresyntax. Hensman reported receipt of support for this manuscript, payment or honoraria and participation on a data safety monitoring or advisory board with Caresyntax. Ramshaw, Schrem, Sharma, Tabiri, and Vibert reported receipt of support for this manuscript, consulting fees and payment or honoraria from Caresyntax. Woods is an employee of Caresyntax.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Adrales, G., Ardito, F., Chowbey, P. et al. Laparoscopic cholecystectomy critical view of safety (LC-CVS): a multi-national validation study of an objective, procedure-specific assessment using video-based assessment (VBA). Surg Endosc 38, 922–930 (2024). https://doi.org/10.1007/s00464-023-10479-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-023-10479-y

Keywords

Search

Navigation