Skip to main content
SpringerLink
Log in

Machine Learning in Laparoscopic Surgery

  • Chapter
  • First Online:
Artificial Intelligence in Medicine

Abstract

Artificial intelligence (AI) refers to the use of computers or machines to mimic human intelligence by learning and autonomously performing complex tasks. Machine learning is a subfield of AI whereby computer algorithms are trained to detect patterns and make predictions based on prior learning without explicit programming. The rise in data availability and improvements in computer power has fuelled rapid development in this emerging technology with broad applications within the health sector. The emerging role and novel applications of machine learning in laparoscopic surgery have been a focus for research in recent years. Potential applications include the autonomous recognition of anatomical structures on a surgical field, intraoperative decision support and alerts. AI algorithms have been trained to track instruments providing feedback on surgical performance to the operating surgeon. In addition, real time awareness of surgical phase has the potential to improve operating room workflow and improved video documentation. Machine learning in laparoscopic surgery remains novel and, despite promising preliminary results, the implementation of these techniques into clinical practice has been limited. This chapter outlines and explores the emerging role of artificial intelligence and machine learning in laparoscopic surgery, summarising the published work and the barriers to implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Garrow CR, Kowalewski KF, Li L, Wagner M, Schmidt MW, Engelhardt S et al (2020) Machine learning for surgical phase recognition: a systematic review. Ann Surg

    Google Scholar 

  2. Hashimoto DA, Rosman G, Rus D, Meireles OR (2018) Artificial intelligence in surgery: promises and perils. Ann Surg 268(1):70–76

    Article  Google Scholar 

  3. Anteby R, Horesh N, Soffer S, Zager Y, Barash Y, Amiel I et al (2021) Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis. Surg Endosc

    Google Scholar 

  4. Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional neural networks: an overview and application in radiology. Insights Imaging 9(4):611–629

    Article  Google Scholar 

  5. Bodenstedt S, Görtler J, Wagner M, Kenngott H, Müller B, Dillmann R et al (2016) Superpixel-based structure classification for laparoscopic surgery, p 978618

    Google Scholar 

  6. Madad Zadeh S, Francois T, Calvet L, Chauvet P, Canis M, Bartoli A et al (2020) SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology. Surg Endosc 29:29

    Google Scholar 

  7. Kletz S, Schoeffmann K, Husslein H (2019) Learning the representation of instrument images in laparoscopy videos. Healthc Technol Lett 6(6):197–203

    Article  Google Scholar 

  8. Lee EJ, Plishker W, Liu X, Bhattacharyya SS, Shekhar R (2019) Weakly supervised segmentation for real-time surgical tool tracking. Healthc Technol Lett 6(6):231–236

    Article  Google Scholar 

  9. Shvets AA, Rakhlin A, Kalinin AA, Iglovikov VI Automatic instrument segmentation in robot-assisted surgery using deep learning. 2018 17th IEEE international conference on machine learning and applications (ICMLA), 17–20 December, 2018. 2018

    Google Scholar 

  10. Jin A, Yeung S, Jopling J, Krause J, Azagury D, Milstein A et al (2018) Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks, pp 691–699

    Google Scholar 

  11. Kamrul Hasan SM, Linte CA (2019) U-NetPlus: a modified encoder-decoder U-net architecture for semantic and instance segmentation of surgical instruments from laparoscopic images. In: Conference proceedings: annual international conference of the IEEE Engineering in Medicine & Biology Society, pp 7205–7211

    Google Scholar 

  12. Nwoye CI, Mutter D, Marescaux J, Padoy N (2019) Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos. Int J Comput Assist Radiol Surg 14(6):1059–1067

    Article  Google Scholar 

  13. Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86–97

    Article  Google Scholar 

  14. Sahu M, Mukhopadhyay A, Szengel A, Zachow S (2016) Tool and phase recognition using contextual CNN features. ArXiv:abs/1610.08854

    Google Scholar 

  15. Fuentes-Hurtado F, Kadkhodamohammadi A, Flouty E, Barbarisi S, Luengo I, Stoyanov D (2019) EasyLabels: weak labels for scene segmentation in laparoscopic videos. Int J Comput Assist Radiol Surg 14(7):1247–1257

    Article  Google Scholar 

  16. Vardazaryan A, Mutter D, Marescaux J, Padoy N (eds) (2018) Weakly-supervised learning for tool localization in laparoscopic videos. Intravascular imaging and computer assisted stenting and large-scale annotation of biomedical data and expert label synthesis. Springer, Cham

    Google Scholar 

  17. Bareum C, Kyungmin J, Songe C, Jaesoon C (2017) Surgical-tools detection based on convolutional neural network in laparoscopic robot-assisted surgery. In: Conference proceedings: annual international conference of the IEEE Engineering in Medicine & Biology Society. 2017, pp 1756–1759

    Google Scholar 

  18. Stauder R, Ostler D, Kranzfelder M, Koller S, Feußner H, Navab N (2016) The TUM LapChole dataset for the M2CAI 2016 workflow challenge. ArXiv:abs/1610.09278

    Google Scholar 

  19. Namazi B, Sankaranarayanan G, Devarajan V (2021) A contextual detector of surgical tools in laparoscopic videos using deep learning. Surg Endosc

    Google Scholar 

  20. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780

    Article  Google Scholar 

  21. Hasan MK, Calvet L, Rabbani N, Bartoli A (2021) Detection, segmentation, and 3D pose estimation of surgical tools using convolutional neural networks and algebraic geometry. Med Image Anal 70:101994

    Article  Google Scholar 

  22. Adams F, Schoelly R, Schlager D, Schoenthaler M, Schoeb DS, Wilhelm K et al (2017) Algorithm-based motion magnification for video processing in urological laparoscopy. J Endourol 31(6):583–587

    Article  Google Scholar 

  23. Akbari H, Kosugi Y, Khorgami Z (2009) Image-guided preparation of the Calot’s triangle in laparoscopic cholecystectomy. In: Conference proceedings: annual international conference of the IEEE Engineering in Medicine & Biology Society, pp 5649–5652

    Google Scholar 

  24. Garcia-Martinez A, Vicente-Samper JM, Sabater-Navarro JM (2017) Automatic detection of surgical haemorrhage using computer vision. Artif Intell Med 78:55–60

    Article  Google Scholar 

  25. Loukas C, Frountzas M, Schizas D (2021) Patch-based classification of gallbladder wall vascularity from laparoscopic images using deep learning. Int J Comput Assist Radiol Surg 16(1):103–113

    Article  Google Scholar 

  26. Prokopetc K, Collins T, Bartoli A (2015) Automatic detection of the uterus and fallopian tube junctions in laparoscopic images. Inf Process Med Imag 24:552–563

    Google Scholar 

  27. Moccia S, Wirkert SJ, Kenngott H, Vemuri AS, Apitz M, Mayer B et al (2018) Uncertainty-aware organ classification for surgical data science applications in laparoscopy. IEEE Trans Biomed Eng 65(11):2649–2659

    Article  Google Scholar 

  28. Petscharnig S, Schöffmann K (2018) Learning laparoscopic video shot classification for gynecological surgery. Multimed Tools Appl 77(7):8061–8079

    Article  Google Scholar 

  29. Hattab G, Arnold M, Strenger L, Allan M, Arsentjeva D, Gold O et al (2020) Kidney edge detection in laparoscopic image data for computer-assisted surgery: kidney edge detection. Int J Comput Assist Radiol Surg 15(3):379–387

    Article  Google Scholar 

  30. Tokuyasu T, Iwashita Y, Matsunobu Y, Kamiyama T, Ishikake M, Sakaguchi S et al (2020) Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc 18:18

    Google Scholar 

  31. Nazir A, Cheema MN, Sheng B, Li P, Li H, Yang P et al (2020) SPST-CNN: spatial pyramid based searching and tagging of liver’s intraoperative live views via CNN for minimal invasive surgery. J Biomed Inform 106:103430

    Article  Google Scholar 

  32. Dergachyova O, Bouget D, Huaulme A, Morandi X, Jannin P (2016) Automatic data-driven real-time segmentation and recognition of surgical workflow. Int J Comput Assist Radiol Surg 11(6):1081–1089

    Article  Google Scholar 

  33. Jin Y, Dou Q, Chen H, Yu L, Qin J, Fu C-W et al (2017) SV-RCNet: workflow recognition from surgical videos using recurrent convolutional network. IEEE Trans Med Imaging:1

    Google Scholar 

  34. Hashimoto DA, Rosman G, Witkowski ER, Stafford C, Navarette-Welton AJ, Rattner DW et al (2019) Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg 270(3):414–421

    Article  Google Scholar 

  35. Volkov M, Hashimoto D, Rosman G, Meireles O, Rus D (2017) Machine learning and coresets for automated real-time video segmentation of laparoscopic and robot-assisted surgery

    Book  Google Scholar 

  36. Jalal N, Alshirbaji T, Möller K (2018) Evaluating convolutional neural network and hidden Markov model for recognising surgical phases in sigmoid resection. Curr Dir Biomed Eng 4:415–418

    Article  Google Scholar 

  37. Kitaguchi D, Takeshita N, Matsuzaki H, Takano H, Owada Y, Enomoto T et al (2019) Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach. Surg Endosc 03:03

    Google Scholar 

  38. Kitaguchi D, Takeshita N, Matsuzaki H, Oda T, Watanabe M, Mori K et al (2020) Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: experimental research. Int J Surg 79:88–94

    Article  Google Scholar 

  39. Madani A, Namazi B, Altieri MS, Hashimoto DA, Rivera AM, Pucher PH et al (2020) Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann Surg

    Google Scholar 

  40. Connor S, Garden OJ (2006) Bile duct injury in the era of laparoscopic cholecystectomy. BJS 93(2):158–168

    Article  Google Scholar 

  41. Jabłońska B, Lampe P (2009) Iatrogenic bile duct injuries: etiology, diagnosis and management. World J Gastroenterol 15(33):4097–4104

    Article  Google Scholar 

  42. Strasberg SM, Brunt LM (2010) Rationale and use of the critical view of safety in laparoscopic cholecystectomy. J Am Coll Surg 211(1):132–138

    Article  Google Scholar 

  43. Mascagni P, Alapatt D, Urade T, Vardazaryan A, Mutter D, Marescaux J et al (2021) A computer vision platform to automatically locate critical events in surgical videos: documenting safety in laparoscopic cholecystectomy. Ann Surg 274:e93–e95

    Article  Google Scholar 

  44. Yengera G, Mutter D, Marescaux J, Padoy N (2018) Less is more: surgical phase recognition with less annotations through self-supervised pre-training of CNN-LSTM networks

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of HPB/UGI Surgery, St Vincent’s Hospital Melbourne, Melbourne, VIC, Australia

    Henry Badgery & Matthew Read

  2. Department of Biomedical Engineering, The University of Melbourne, Parkville, VIC, Australia

    Yuning Zhou & Catherine Davey

  3. Department of Surgery, The University of Melbourne, St Vincent’s Hospital, Melbourne, VIC, Australia

    Athanasia Siderellis & Matthew Read

Authors
  1. Henry Badgery
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuning Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Athanasia Siderellis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matthew Read
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Catherine Davey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Henry Badgery .

Editor information

Editors and Affiliations

  1. Department of Emergency Medicine, Monash Health, Clayton, VIC, Australia

    Manda Raz

  2. St Vincent’s Hospital Melbourne, Fitzroy, VIC, Australia

    Tam C. Nguyen

  3. St. Vincent’s Health Australia, Sydney, NSW, Australia

    Erwin Loh

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Badgery, H., Zhou, Y., Siderellis, A., Read, M., Davey, C. (2022). Machine Learning in Laparoscopic Surgery. In: Raz, M., Nguyen, T.C., Loh, E. (eds) Artificial Intelligence in Medicine. Springer, Singapore. https://doi.org/10.1007/978-981-19-1223-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-981-19-1223-8_8

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-19-1222-1

  • Online ISBN: 978-981-19-1223-8

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation