Skip to main content

Advertisement

SpringerLink
Log in

An intraoperative artificial intelligence system identifying anatomical landmarks for laparoscopic cholecystectomy: a prospective clinical feasibility trial (J-SUMMIT-C-01)

  • Original Article
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

We have implemented Smart Endoscopic Surgery (SES), a surgical system that uses artificial intelligence (AI) to detect the anatomical landmarks that expert surgeons base on to perform certain surgical maneuvers. No report has verified the use of AI-based support systems for surgery in clinical practice, and no evaluation method has been established. To evaluate the detection performance of SES, we have developed and established a new evaluation method by conducting a clinical feasibility trial.

Methods

A single-center prospective clinical feasibility trial was conducted on 10 cases of LC performed at Oita University hospital. Subsequently, an external evaluation committee (EEC) evaluated the AI detection accuracy for each landmark using five-grade rubric evaluation and DICE coefficient. We defined LM-CBD as the expert surgeon’s “judge” of the cystic bile duct in endoscopic images.

Results

The average detection accuracy on the rubric by the EEC was 4.2 ± 0.8 for the LM-CBD. The DICE coefficient between the AI detection area of the LM-CBD and the EEC members’ evaluation was similar to the mean value of the DICE coefficient between the EEC members. The DICE coefficient was high score for the case that was highly evaluated by the EEC on a five-grade scale.

Conclusion

This is the first feasible clinical trial of an AI system designed for intraoperative use and to evaluate the AI system using an EEC. In the future, this concept of evaluation for the AI system would contribute to the development of new AI navigation systems for surgery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Turing AM (1950) Computing machinery and intelligence. Mind 236:433–460

    Article  Google Scholar 

  2. Yu KH, Beam AL, Kohane IS (2018) Artificial intelligence in healthcare. Nat Biomed Eng 2:719–731

    Article  PubMed  Google Scholar 

  3. Reynolds W Jr (2001) The first laparoscopic cholecystectomy. JSLS 5:89–94

    PubMed  PubMed Central  Google Scholar 

  4. Madani A, Namazi B, Altieri MS, Hashimoto DA, Rivera AM, Pucher PH, Navarrete-Welton A, Sankaranarayanan G, Brunt LM, Okrainec A, Alseidi A (2020) Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann Surg 276(2):363–369

    Article  PubMed  Google Scholar 

  5. Iwashita Y, Hibi T, Ohyama T, Umezawa A, Takada T, Strasberg SM, Asbun HJ, Pitt HA, Han HS, Hwang TL, Suzuki K, Yoon YS, Choi IS, Yoon DS, Huang WS, Yoshida M, Wakabayashi G, Miura F, Okamoto K, Endo I, de Santibañes E, Giménez ME, Windsor JA, Garden OJ, Gouma DJ, Cherqui D, Belli G, Dervenis C, Deziel DJ, Jonas E, Jagannath P, Supe AN, Singh H, Liau KH, Chen XP, Chan ACW, Lau WY, Fan ST, Chen MF, Kim MH, Honda G, Sugioka A, Asai K, Wada K, Mori Y, Higuchi R, Misawa T, Watanabe M, Matsumura N, Rikiyama T, Sata N, Kano N, Tokumura H, Kimura T, Kitano S, Inomata M, Hirata K, Sumiyama Y, Inui K, Yamamoto M (2017) Delphi consensus on bile duct injuries during laparoscopic cholecystectomy: an evolutionary cul-de-sac or the birth pangs of a new technical framework? J Hepatobiliary Pancreat Sci 24:591–602

    Article  PubMed  Google Scholar 

  6. Davidoff AM, Pappas TN, Murray EA, Hilleren DJ, Johnson RD, Baker ME, Newman GE, Cotton PB, Meyers WC (1992) Mechanisms of major biliary injury during laparoscopic cholecystectomy. Ann Surg 215:196–202

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Hugh TB (2002) New strategies to prevent laparoscopic bile duct injury—surgeons can learn from pilots. Surgery 132:826–835

    Article  PubMed  Google Scholar 

  8. Inomata M, Shiroshita H, Uchida H, Bandoh T, Akira S, Yamaguchi S, Kurokawa Y, Seki Y, Eguchi S, Wada N, Takiguchi S, Ieiri S, Endo S, Iwazaki M, Sato Y, Tamaki Y, Kitamura K, Tabata M, Kanayama H, Mimata H, Hasegawa T, Takahashi H, Onishi K, Uemura T, Hashizume M, Matsumoto S, Kitano S, Watanabe M (2020) Current status of endoscopic surgery in Japan: the 14th national survey of endoscopic surgery by the Japan society for endoscopic surgery. Asian J Endosc Surg 13:7–18

    Article  PubMed  Google Scholar 

  9. Navez B, Ungureanu F, Michiels M, Claeys D, Muysoms F, Hubert C, Vanderveken M, Detry O, Detroz B, Closset J, Devos B, Kint M, Navez J, Zech F, Gigot JF, Belgian Group for Endoscopic Surgery (BGES) and the Hepatobiliary and Pancreatic Section (HBPS) of the Royal Belgian Society of Surgery (2012) Surgical management of acute cholecystitis: results of a 2-year prospective multicenter survey in Belgium. Surg Endosc 26:2436–2445

    Article  PubMed  Google Scholar 

  10. Pucher PH, Brunt LM, Fanelli RD, Asbun HJ, Aggarwal R (2015) SAGES expert delphi consensus: critical factors for safe surgical practice in laparoscopic cholecystectomy. Surg Endosc 29:3074–3085

    Article  PubMed  Google Scholar 

  11. Harboe KM, Bardram L (2011) The quality of cholecystectomy in Denmark: outcome and risk factors for 20,307 patients from the national database. Surg Endosc 25:1630–1641

    Article  PubMed  Google Scholar 

  12. Soper NJ, Stockmann PT, Dunnegan DL, Ashley SW (1992) Laparoscopic cholecystectomy the new ‘gold standard’? Arch Surg 127:917–921

    Article  CAS  PubMed  Google Scholar 

  13. Connor SJ, Perry W, Nathanson L, Hugh TB, Hugh TJ (2014) Using a standardized method for laparoscopic cholecystectomy to create a concept operation-specific checklist. HPB (Oxford) 16:422–429

    Article  PubMed  Google Scholar 

  14. Way LW, Stewart L, Gantert W, Liu K, Lee CM, Whang K, Hunter JG (2003) Causes and prevention of laparoscopic bile duct injuries: analysis of 252 cases from a human factors and cognitive psychology perspective. Ann Surg 237:460–469

    Article  PubMed  PubMed Central  Google Scholar 

  15. Tokuyasu T, Iwashita Y, Matsunobu Y, Kamiyama T, Ishikake M, Sakaguchi S, Ebe K, Tada K, Endo Y, Etoh T, Nakashima M, Inomata M (2021) Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc 35:1651–1658

    Article  PubMed  Google Scholar 

  16. de la Fuente LE, Muñoz García Á, Santos del Blanco L, Fraile Marinero JC, Pérez Turiel J (2020) Automatic gauze tracking in laparoscopic surgery using image texture analysis. Comput Methods Prog Biomed 190:105378

    Article  Google Scholar 

  17. Madad Zadeh S, François T, Calvet L, Chauvet P, Canis M, Bartoli A, Bourdel N (2020) SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology. Surg Endosc 34:5377–5383

    Article  PubMed  Google Scholar 

  18. Anteby R, Horesh N, Soffer S, Zager Y, Barash Y, Amiel I, Rosin D, Gutman M, Klang E (2021) Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis. Surg Endosc 35:1521–1533

    Article  PubMed  Google Scholar 

  19. Feldman LS, Pryor AD, Gardner AK, Dunkin BJ, Schultz L, Awad MM, Ritter EM (2020) SAGES video-based assessment (VBA) program: a vision for life-long learning for surgeons. Surg Endosc 34:3285–3288

    Article  PubMed  Google Scholar 

  20. Shamir RR, Duchin Y, Kim J, Sapiro G, Harel N (2018) Continuous dice coefficient: a method for evaluating probabilistic segmentations. bioRxiv. 306977

  21. Sudre CH, Li W, Vercauteren T, Ourselin S, Cardoso MJ (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. Springer International Publishing, Cham

    Book  Google Scholar 

  22. Strasberg SM (2002) Avoidance of biliary injury during laparoscopic cholecystectomy. J Hepatobiliary Pancreat Surg 9:543–547

    Article  PubMed  Google Scholar 

  23. Mascagni P, Vardazaryan A, Alapatt D, Urade T, Emre T, Fiorillo C, Pessaux P, Mutter D, Marescaux J, Costamagna G, Dallemagne B, Padoy N (2020) Artificial intelligence for surgical safety: automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning. Ann Surg. https://doi.org/10.1097/SLA.0000000000004351

    Article  PubMed  Google Scholar 

  24. Luo H, Xu G, Li C, He L, Luo L, Wang Z, Jing B, Deng Y, Jin Y, Li Y, Li B, Tan W, He C, Seeruttun S, Wu Q, Huang J, Huang D, Chen B, Lin S, Chen Q, Yuan C, Chen H, Pu H, Zhou F, He Y, Xu R (2019) Real-time artificial intelligence for detection of upper gastrointestinal cancer by endoscopy: a multicentre, case-control, diagnostic study. Lancet Oncol 20:1645–1654

    Article  CAS  PubMed  Google Scholar 

  25. Nakajima K, Inomata M, Akagi T, Etoh T, Sugihara K, Watanabe M, Yamamoto S, Katayama H, Moriya Y, Kitano S (2014) Quality control by photo documentation for evaluation of laparoscopic and open colectomy with D3 resection for stage II/III colorectal cancer: Japan clinical oncology group study JCOG 0404. Jpn J Clin Oncol 44:799–806

    Article  PubMed  Google Scholar 

  26. Kletz S, Schoeffmann K, Husslein H (2019) Learning the representation of instrument images in laparoscopy videos. Healthc Technol Lett 6:197–203

    Article  PubMed  PubMed Central  Google Scholar 

  27. Kitaguchi D, Takeshita N, Matsuzaki H, Hasegawa H, Igaki T, Oda T, Ito M (2021) Deep learning-based automatic surgical step recognition in intraoperative videos for transanal total mesorectal excision. Surg Endosc 36:1143–1151

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank the patients who participated in this study, and the many healthcare and social care professionals who contributed. Yukio Iwashita, Kazuhiro Tada, and Kiminori Watanabe supervised the creation of training datasets and participated as surgeons in the clinical feasibility trial. We would like to thank Dr. Koji Asai of Toho University Medical Center Ohashi Hospital, Dr. Yasutoshi Mori of University of Occupational and Environmental Health, Japan, and Dr. Toshio Bando of Oita Prefectural Hospital, who conducted an objective evaluation of our clinical feasibility trial as external evaluation committee members. The experimental machine operation team of Olympus Corporation operated the AI-equipped workstation in the operating room. The authors vouch for the accuracy and completeness of the data and for fidelity to the protocol. Nozomi Komine performed office work, including document preparation and schedule adjustment in carrying out this research. We would like to thank Editage (www.editage.com) for English language editing.

Funding

The study was supported by Japan Agency for Medical Research and Development (grant number: 20he2302003h0202).

Author information

Authors and Affiliations

  1. Department of Gastroenterological and Pediatric Surgery, Faculty of Medicine, Oita University, 1-1 Idaigaoka, Hasama-machi, Yufu, Oita, 879-5593, Japan

    Hiroaki Nakanuma, Yuichi Endo, Atsuro Fujinaga, Masahiro Kawamura, Takahide Kawasaki, Takashi Masuda, Teijiro Hirashita, Tsuyoshi Etoh & Masafumi Inomata

  2. Department of Information System and Engineering, Fukuoka Institute of Technology, Fukuoka, Japan

    Ken’ichi Shinozuka, Yusuke Matsunobu & Tatsushi Tokuyasu

  3. R&D, Customer Solutions Development, Platform Technology, Olympus Technologies Asia, Hachioji, Japan

    Toshiya Kamiyama, Makoto Ishikake & Kohei Ebe

Authors
  1. Hiroaki Nakanuma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuichi Endo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Atsuro Fujinaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masahiro Kawamura
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Takahide Kawasaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Takashi Masuda
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Teijiro Hirashita
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tsuyoshi Etoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ken’ichi Shinozuka
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Yusuke Matsunobu
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Toshiya Kamiyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Makoto Ishikake
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Kohei Ebe
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Tatsushi Tokuyasu
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Masafumi Inomata
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hiroaki Nakanuma.

Ethics declarations

Disclosures

HN, YE, AF, MK, TK, TM, TH, and TE have no conflicts of interest or financial ties to disclose. KS, YM, TK, MI, and KE have no conflicts of interest or financial ties to disclose. TT and MI have no conflicts of interest or financial ties to disclose. The patent for the tile-shaped display system for the target detection algorithm is pending (application no. PCT/JP2021/2754).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 14 KB)

Supplementary file2 (DOCX 17 KB)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nakanuma, H., Endo, Y., Fujinaga, A. et al. An intraoperative artificial intelligence system identifying anatomical landmarks for laparoscopic cholecystectomy: a prospective clinical feasibility trial (J-SUMMIT-C-01). Surg Endosc 37, 1933–1942 (2023). https://doi.org/10.1007/s00464-022-09678-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-022-09678-w

Keywords

Search

Navigation