Skip to main content

Advertisement

SpringerLink
Log in

Validation of an artificial intelligence platform for the guidance of safe laparoscopic cholecystectomy

  • 2022 SAGES Oral
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Many surgical adverse events, such as bile duct injuries during laparoscopic cholecystectomy (LC), occur due to errors in visual perception and judgment. Artificial intelligence (AI) can potentially improve the quality and safety of surgery, such as through real-time intraoperative decision support. GoNoGoNet is a novel AI model capable of identifying safe (“Go”) and dangerous (“No-Go”) zones of dissection on surgical videos of LC. Yet, it is unknown how GoNoGoNet performs in comparison to expert surgeons. This study aims to evaluate the GoNoGoNet’s ability to identify Go and No-Go zones compared to an external panel of expert surgeons.

Methods

A panel of high-volume surgeons from the SAGES Safe Cholecystectomy Task Force was recruited to draw free-hand annotations on frames of prospectively collected videos of LC to identify the Go and No-Go zones. Expert consensus on the location of Go and No-Go zones was established using Visual Concordance Test pixel agreement. Identification of Go and No-Go zones by GoNoGoNet was compared to expert-derived consensus using mean F1 Dice Score, and pixel accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).

Results

A total of 47 frames from 25 LC videos, procured from 3 countries and 9 surgeons, were annotated simultaneously by an expert panel of 6 surgeons and GoNoGoNet. Mean (± standard deviation) F1 Dice score were 0.58 (0.22) and 0.80 (0.12) for Go and No-Go zones, respectively. Mean (± standard deviation) accuracy, sensitivity, specificity, PPV and NPV for the Go zones were 0.92 (0.05), 0.52 (0.24), 0.97 (0.03), 0.70 (0.21), and 0.94 (0.04) respectively. For No-Go zones, these metrics were 0.92 (0.05), 0.80 (0.17), 0.95 (0.04), 0.84 (0.13) and 0.95 (0.05), respectively.

Conclusions

AI can be used to identify safe and dangerous zones of dissection within the surgical field, with high specificity/PPV for Go zones and high sensitivity/NPV for No-Go zones. Overall, model prediction was better for No-Go zones compared to Go zones. This technology may eventually be used to provide real-time guidance and minimize the risk of adverse events.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. World Health Organization (Genève). World Alliance for Patient Safety (2009) WHO Guidelines for Safe Surgery 2009: Safe Surgery Saves Lives, General Directorate of Health of Portugal

    Google Scholar 

  2. Rogers SO Jr, Gawande AA, Kwaan M, Puopolo AL, Yoon C, Brennan TA, Studdert DM (2006) Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery 140:25–33

    Article  PubMed  Google Scholar 

  3. Madani A, Vassiliou MC, Watanabe Y, Al-Halabi B, Al-Rowais MS, Deckelbaum DL, Fried GM, Feldman LS (2017) What are the principles that guide behaviors in the operating room?: creating a framework to define and measure performance. Ann Surg 265:255–267

    Article  PubMed  Google Scholar 

  4. Leape LL et al (1992) The nature of adverse events in hospitalized patients results of the harvard medical practice study II. Am J Infect Control 20:332

    Article  Google Scholar 

  5. Gawande AA, Zinner MJ, Studdert DM, Brennan TA (2003) Analysis of errors reported by surgeons at three teaching hospitals. Surgery 133:614–621

    Article  PubMed  Google Scholar 

  6. Madani A, Namazi B, Altieri MS, Hashimoto DA, Rivera AM, Pucher PH, Navarrete-Welton A, Sankaranarayanan G, Brunt LM, Okrainec A, Alseidi A (2020) Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann Surg. https://doi.org/10.1097/SLA.0000000000004594

    Article  PubMed  Google Scholar 

  7. Mascagni P, Alapatt D, Urade T, Vardazaryan A, Mutter D, Marescaux J, Costamagna G, Dallemagne B, Padoy N (2021) A computer vision platform to automatically locate critical events in surgical videos: documenting safety in laparoscopic cholecystectomy. Ann Surg 274:e93–e95

    Article  PubMed  Google Scholar 

  8. Tokuyasu T, Iwashita Y, Matsunobu Y, Kamiyama T, Ishikake M, Sakaguchi S, Ebe K, Tada K, Endo Y, Etoh T, Nakashima M, Inomata M (2021) Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc 35:1651–1658

    Article  PubMed  Google Scholar 

  9. Madani A, Watanabe Y, Bilgic E, Pucher PH, Vassiliou MC, Aggarwal R, Fried GM, Mitmaker EJ, Feldman LS (2017) Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool. Surg Endosc 31:1203–1212

    Article  PubMed  Google Scholar 

  10. Mascagni P, Vardazaryan A, Alapatt D, Urade T, Emre T, Fiorillo C, Pessaux P, Mutter D, Marescaux J, Costamagna G, Dallemagne B, Padoy N (2020) Artificial intelligence for surgical safety: automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning. Ann Surg. https://doi.org/10.1097/SLA.0000000000004351

    Article  PubMed  Google Scholar 

  11. Hashimoto DA, Rosman G, Witkowski ER, Stafford C, Navarette-Welton AJ, Rattner DW, Lillemoe KD, Rus DL, Meireles OR (2019) Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg 270:414–421

    Article  PubMed  Google Scholar 

  12. Ward TM, Hashimoto DA, Ban Y, Rosman G, Meireles OR (2022) Artificial intelligence prediction of cholecystectomy operative course from automated identification of gallbladder inflammation. Surg Endosc. https://doi.org/10.1007/s00464-022-09009-z

    Article  PubMed  PubMed Central  Google Scholar 

  13. Way LW, Stewart L, Gantert W, Liu K, Lee CM, Whang K, Hunter JG (2003) Causes and prevention of laparoscopic bile duct injuries. Ann Surg 237:460–469

    Article  PubMed  PubMed Central  Google Scholar 

  14. Brunt LM, Deziel DJ, Telem DA, Strasberg SM, Aggarwal R, Asbun H, Bonjer J, McDonald M, Alseidi A, Ujiki M, Riall TS, Hammill C, Moulton C-A, Pucher PH, Parks RW, Ansari MT, Connor S, Dirks RC, Anderson B, Altieri MS, Tsamalaidze L, Stefanidis D, and the Prevention of Bile Duct Injury Consensus Work Group (2020) Safe cholecystectomy multi-society practice guideline and state of the art consensus conference on prevention of bile duct injury during cholecystectomy. Ann Surg 272:3–23

    Article  PubMed  Google Scholar 

  15. Madani A, Watanabe Y, Feldman LS, Vassiliou MC, Barkun JS, Fried GM, Aggarwal R (2015) Expert intraoperative judgment and decision-making: defining the cognitive competencies for safe laparoscopic cholecystectomy. J Am Coll Surg 221:931-940.e8

    Article  PubMed  Google Scholar 

  16. Madani A, Gornitsky J, Watanabe Y, Benay C, Altieri MS, Pucher PH, Tabah R, Mitmaker EJ (2018) Measuring decision-making during thyroidectomy: validity evidence for a web-based assessment tool. World J Surg 42:376–383

    Article  PubMed  Google Scholar 

  17. Madani A, Grover K, Watanabe Y (2020) Measuring and teaching intraoperative decision-making using the visual concordance test: deliberate practice of advanced cognitive skills. JAMA Surg 155:78–79

    Article  PubMed  Google Scholar 

  18. Sudre CH, Li W, Vercauteren T, Ourselin S, Jorge Cardoso M (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. Deep Learn Med Image Anal Multimodal Learn Clin Decis Support 2017(240):248

    Google Scholar 

  19. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  20. Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56

    Article  CAS  PubMed  Google Scholar 

  21. Miller DD, Douglas Miller D, Brown EW (2018) Artificial intelligence in medical practice: the question to the answer? Am J Med 131:129–133

    Article  PubMed  Google Scholar 

  22. Flum DR, Cheadle A, Prela C, Dellinger EP, Chan L (2003) Bile duct injury during cholecystectomy and survival in medicare beneficiaries. JAMA 290:2168–2173

    Article  CAS  PubMed  Google Scholar 

  23. Ejaz A, Spolverato G, Kim Y, Dodson R, Sicklick JK, Pitt HA, Lillemoe KD, Cameron JL, Pawlik TM (2014) Long-term health-related quality of life after iatrogenic bile duct injury repair. J Am Coll Surg 219:923–32.e10

    Article  PubMed  Google Scholar 

  24. Schwaitzberg SD, Scott DJ, Jones DB, McKinley SK, Castrillion J, Hunter TD, Michael Brunt L (2014) Threefold increased bile duct injury rate is associated with less surgeon experience in an insurance claims database. Surg Endosc 28:3068–3073

    Article  PubMed  Google Scholar 

  25. Pucher PH, Brunt LM, Davies N, Linsk A, Munshi A, Rodriguez HA, Fingerhut A, Fanelli RD, Asbun H, Aggarwal R, Safe Cholecystectomy Task Force SAGES (2018) Outcome trends and safety measures after 30 years of laparoscopic cholecystectomy: a systematic review and pooled data analysis. Surg Endosc 32:2175–2183

    Article  PubMed  PubMed Central  Google Scholar 

  26. Michael Brunt L, Deziel DJ, Telem DA, Strasberg SM, Aggarwal R, Asbun H, Bonjer J, McDonald M, Alseidi A, Ujiki M, Riall TS, Hammill C, Moulton C-A, Pucher PH, Parks RW, Ansari MT, Connor S, Dirks RC, Anderson B, Altieri MS, Tsamalaidze L, Stefanidis D, Prevention of Bile Duct Injury Consensus Work Group (2020) Safe cholecystectomy multi-society practice guideline and state-of-the-art consensus conference on prevention of bile duct injury during cholecystectomy. Surg Endosc 34:2827–2855

    Article  Google Scholar 

  27. Way LW, Stewart L, Gantert W, Liu K, Lee CM, Whang K, Hunter JG (2003) Causes and prevention of laparoscopic bile duct injuries: analysis of 252 cases from a human factors and cognitive psychology perspective. Ann Surg 237:460–469

    Article  PubMed  PubMed Central  Google Scholar 

  28. Kable AK, Gibberd RW, Spigelman AD (2002) Adverse events in surgical patients in Australia. Int J Qual Health Care 14:269–276

    Article  CAS  PubMed  Google Scholar 

  29. Gawande AA, Thomas EJ, Zinner MJ, Brennan TA (1999) The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 126:66–75

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We acknowledge the work of Mr. Robert Messina for developing the Think Like A Surgeon annotation tool.

Funding

SAGES Research Grant Award.

Author information

Authors and Affiliations

  1. Surgical Artificial Intelligence Research Academy, University Health Network, Toronto, ON, Canada

    Simon Laplante, Parmiss Kiani, Allan Okrainec & Amin Madani

  2. Department of Surgery, University of Toronto, Toronto, ON, Canada

    Simon Laplante, Allan Okrainec & Amin Madani

  3. Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX, USA

    Babak Namazi

  4. Department of Surgery, University Hospitals, Cleveland, OH, USA

    Daniel A. Hashimoto

  5. Department of Surgery, University of California, San Francisco, CA, USA

    Adnan Alseidi

  6. Instituto de Gastroenterologia Boliviano Japones, Cochabamba, Bolivia

    Mauricio Pasten

  7. Department of Surgery, Washington University School of Medicine, St. Louis, MO, USA

    L. Michael Brunt

  8. Department of Surgery, Northeast Georgia Medical Center, Georgia, USA

    Sujata Gill

  9. Department of Surgery, Texas Tech Paul L Foster School of Medicine, El Paso, TX, USA

    Brian Davis

  10. Department of Surgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA

    Matthew Bloom

  11. Department of Surgery, Boston medical center, Boston, MA, USA

    Luise Pernar

  12. MIS Fellow, Toronto Western Hospital, Division of General Surgery, 8MP-325., 399 Bathurst St, Toronto,, ON, M5T 2S8, Canada

    Simon Laplante

Authors
  1. Simon Laplante
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Babak Namazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Parmiss Kiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel A. Hashimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adnan Alseidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mauricio Pasten
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. L. Michael Brunt
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sujata Gill
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Brian Davis
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Matthew Bloom
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Luise Pernar
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Allan Okrainec
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Amin Madani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Laplante.

Ethics declarations

Disclosures

Drs. Amin Madani is a consultant for Activ Surgical. Dr. Daniel Hashimoto is a consultant for Activ Surgical, Johnson & Johnson Institute, Verily Life Sciences, Worrell, and Mosaic Research Management. Dr. Allan Okrainec has received honoraria for speaking and teaching from Medtronic, Ethicon, and Merck. Parmiss Kiani has no disclosures. Drs. Simon Laplante, Babak Namazi, Brian Davis, Mauricio Pasten, L. Michael Brunt, Adnan Alseidi, Luise Pernar, Matthew Bloom and Sujata Gill, have no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 2299 kb)

Supplementary file2 (MP4 18549 kb)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Laplante, S., Namazi, B., Kiani, P. et al. Validation of an artificial intelligence platform for the guidance of safe laparoscopic cholecystectomy. Surg Endosc 37, 2260–2268 (2023). https://doi.org/10.1007/s00464-022-09439-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-022-09439-9

Keywords

Search

Navigation