Advertisement

Artificial intelligence and robotics: a combination that is changing the operating room

  • Iulia Andras
  • Elio Mazzone
  • Fijs W. B. van Leeuwen
  • Geert De Naeyer
  • Matthias N. van Oosterom
  • Sergi Beato
  • Tessa Buckle
  • Shane O’Sullivan
  • Pim J. van Leeuwen
  • Alexander Beulens
  • Nicolae Crisan
  • Frederiek D’Hondt
  • Peter Schatteman
  • Henk van Der Poel
  • Paolo Dell’OglioEmail author
  • Alexandre Mottrie
Topic Paper

Abstract

Purpose

The aim of the current narrative review was to summarize the available evidence in the literature on artificial intelligence (AI) methods that have been applied during robotic surgery.

Methods

A narrative review of the literature was performed on MEDLINE/Pubmed and Scopus database on the topics of artificial intelligence, autonomous surgery, machine learning, robotic surgery, and surgical navigation, focusing on articles published between January 2015 and June 2019. All available evidences were analyzed and summarized herein after an interactive peer-review process of the panel.

Literature review

The preliminary results of the implementation of AI in clinical setting are encouraging. By providing a readout of the full telemetry and a sophisticated viewing console, robot-assisted surgery can be used to study and refine the application of AI in surgical practice. Machine learning approaches strengthen the feedback regarding surgical skills acquisition, efficiency of the surgical process, surgical guidance and prediction of postoperative outcomes. Tension-sensors on the robotic arms and the integration of augmented reality methods can help enhance the surgical experience and monitor organ movements.

Conclusions

The use of AI in robotic surgery is expected to have a significant impact on future surgical training as well as enhance the surgical experience during a procedure. Both aim to realize precision surgery and thus to increase the quality of the surgical care. Implementation of AI in master–slave robotic surgery may allow for the careful, step-by-step consideration of autonomous robotic surgery.

Keywords

Artificial intelligence Autonomous surgery Machine learning Robotic surgery Surgical navigation 

Notes

Acknowledgements

This research was conducted with the support of the European Urological Scholarship Programme and an NWO TTW VICI grant (TTW BTG 16141).

Authors contribution

Protocol/project development: IA, EM, PDO, and AM. Data collection or management: IA, EM, and PDO. Data analysis: IA, EM, and PDO. Manuscript writing: IA, EM, and PDO. Manuscript editing: FWBL, GN, MNO, SB, TB, SOS, PJL, AB, NC, FDH, PS, HDP, and AM.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflicts of interest.

References

  1. 1.
    Bachman AG, Parker AA, Shaw MD et al (2017) Minimally invasive versus open approach for cystectomy: trends in the utilization and demographic or clinical predictors using the National Cancer Database. Urology 103:99–105PubMedCrossRefPubMedCentralGoogle Scholar
  2. 2.
    Mazzone E, Mistretta FA, Knipper S et al (2019) Contemporary North-American assessment of robot-assisted surgery rates and total hospital charges for major surgical uro-oncological procedures. J Endourol 33(6):438–447PubMedCrossRefPubMedCentralGoogle Scholar
  3. 3.
    Autorino R, Porpiglia F, Dasgupta P et al (2017) Precision surgery and genitourinary cancers. Eur J Surg Oncol 43(5):893–908PubMedCrossRefPubMedCentralGoogle Scholar
  4. 4.
    Veronesi U, Stafyla V, Luini A, Veronesi P (2012) Breast cancer: from “maximum tolerable” to “minimum effective” treatment. Front Oncol 2:125PubMedPubMedCentralGoogle Scholar
  5. 5.
    Gallagher AG (2018) Proficiency-based progression simulation training for more than an interesting educational experience. J Musculoskelet Surg Res 2:139–141CrossRefGoogle Scholar
  6. 6.
    Satava RM, Stefanidis D, Levy JS et al (2019) Proving the effectiveness of the fundamentals of robotic surgery (FRS) skills curriculum: a single-blinded, multispecialty, multi-institutional randomized control trial. Ann Surg.  https://doi.org/10.1097/SLA.0000000000003220 CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Wang D, Khosla A, Gargeya R et al (2016) Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718Google Scholar
  8. 8.
    Bergquist S, Brooks G, Keating N et al (2017) Classifying lung cancer severity with ensemble machine learning in health care claims data. Proc Mach Learn Res 68:25–38PubMedPubMedCentralGoogle Scholar
  9. 9.
    Hashimoto DA, Rosman G, Rus D, Meireles OR (2018) Artificial intelligence in surgery: promises and perils. Ann Surg 268(1):70–76PubMedPubMedCentralCrossRefGoogle Scholar
  10. 10.
    Shouval R, Hadanny A, Shlomo N et al (2017) Machine learning for prediction of 30-day mortality after ST elevation myocardial infraction: an acute coronary syndrome Israeli survey data mining study. Int J Cardiol 246:7–13PubMedCrossRefPubMedCentralGoogle Scholar
  11. 11.
    Kassahun Y, Yu B, Tibebu AT et al (2016) Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int J Comput Assist Radiol Surg 11(4):553–568PubMedCrossRefPubMedCentralGoogle Scholar
  12. 12.
    Fard MJ, Ameri S, Darin Ellis R et al (2018) Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int J Med Robot 14(1):e1850CrossRefGoogle Scholar
  13. 13.
    Wang Z, Majewicz Fey A (2018) Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. Int J Comput Assist Radiol Surg 13(12):1959–1970PubMedCrossRefPubMedCentralGoogle Scholar
  14. 14.
    Ershad M, Rege R, Majewicz Fey A (2019) Automatic and near real-time stylistic behavior assessment in robotic surgery. Int J Comput Assist Radiol Surg 14(4):635–643PubMedCrossRefGoogle Scholar
  15. 15.
    Hung AJ, Chen J, Che Z et al (2018) Utilizing machine learning and automated performance metrics to evaluate robot-assisted radical prostatectomy performance and predict outcomes. J Endourol 32(5):438–444PubMedCrossRefGoogle Scholar
  16. 16.
    Dai Y, Abiri A, Pensa J et al (2019) Biaxial sensing suture breakage warning system for robotic surgery. Biomed Microdevices 21(1):10PubMedPubMedCentralCrossRefGoogle Scholar
  17. 17.
    Zhao B, Waterman RS, Urman RD, Gabriel RA (2019) A machine learning approach to predicting case duration for robot-assisted surgery. J Med Syst 43(2):32PubMedCrossRefGoogle Scholar
  18. 18.
    Hung AJ, Chen J, Ghodoussipour S et al (2019) A deep-learning model using automated performance metrics and clinical features to predict urinary continence recovery after robot-assisted radical prostatectomy. BJU Int.  https://doi.org/10.1111/bju.14735 CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Collins JW, Dell'Oglio P, Hung AJ, Brook NR (2018) The importance of technical and non-technical skills in robotic surgery training. Eur Urol Focus 4(5):674–676PubMedCrossRefPubMedCentralGoogle Scholar
  20. 20.
    Lovegrove CE, Elhage O, Khan MS et al (2017) Training modalities in robot-assisted urologic surgery: a systematic review. Eur Urol Focus 3(1):102–116PubMedCrossRefPubMedCentralGoogle Scholar
  21. 21.
    Mazzone E, Dell’Oglio P, Mottrie A (2019) Outcomes report of the first ERUS robotic urology curriculum-trained surgeon in Turkey: the importance of structured and validated training programs for global outcomes improvement. Turk J Urol 45(3):189–190PubMedCentralCrossRefGoogle Scholar
  22. 22.
    Mottrie A, Novara G, van der Poel H et al (2016) The European Association of Urology robotic training curriculum: an update. Eur Urol Focus 2(1):105–108PubMedCrossRefPubMedCentralGoogle Scholar
  23. 23.
    Larcher A, De Naeyer G, Turri F et al (2019) The ERUS curriculum for robot-assisted partial nephrectomy: structure definition and pilot clinical validation. Eur Urol 75(6):1023–1031PubMedCrossRefPubMedCentralGoogle Scholar
  24. 24.
    Dell’Oglio P, Turri F, Larcher A et al (2019) Definition of a structured training curriculum for robot-assisted radical cystectomy: a Delphi-consensus study led by the ERUS Educational Board. Eur Urol Suppl 18(1):e1116–e1119CrossRefGoogle Scholar
  25. 25.
    Chen J, Cheng N, Cacciamani G et al (2019) Objective assessment of robotic surgical technical skill: a systematic review. J Urol 201(3):461–469PubMedCrossRefPubMedCentralGoogle Scholar
  26. 26.
    Schout BMA, Hendrikx AJM, Scheele F, Bemelmans BLH, Scherpbier AJJA (2010) Validation and implementation of surgical simulators: a critical review of present, past, and future. Surg Endosc 24(3):536–546PubMedCrossRefPubMedCentralGoogle Scholar
  27. 27.
    Goldenberg MG, Lee JY, Kwong JCC, Grantcharov TP, Costello A (2018) Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence. BJU Int 122(3):501–519PubMedCrossRefPubMedCentralGoogle Scholar
  28. 28.
    Ganni S, Botden SMBI, Chmarra M, Goossens RHM, Jakimowicz JJ (2018) A software-based tool for video motion tracking in the surgical skills assessment landscape. Surg Endosc 32(6):2994–2999PubMedPubMedCentralCrossRefGoogle Scholar
  29. 29.
    Hung AJ, Chen J, Gill IS (2018) Automated performance metrics and machine learning algorithms to measure surgeon performance and anticipate clinical outcomes in robotic surgery. JAMA Surg 153(8):770–771CrossRefGoogle Scholar
  30. 30.
    Delto JC, Wayne G, Yanes R, Nieder AM, Bhandari A (2015) Reducing robotic prostatectomy costs by minimizing instrumentation. J Endourol 29(5):556–560PubMedCrossRefPubMedCentralGoogle Scholar
  31. 31.
    Ramirez D, Ganesan V, Nelson RJ, Haber GP (2016) Reducing costs for robotic radical prostatectomy: three-instrument technique. Urology 95:213–215PubMedCrossRefPubMedCentralGoogle Scholar
  32. 32.
    Basto M, Sathianathen N, Te Marvelde L et al (2016) Patterns-of-care and health economic analysis of robot-assisted radical prostatectomy in the Australian public health system. BJU Int 117(6):930–939PubMedCrossRefPubMedCentralGoogle Scholar
  33. 33.
    Pandit JJ, Carey A (2006) Estimating the duration of common elective operations: implications for operating list management. Anaesthesia 61:768–776PubMedCrossRefPubMedCentralGoogle Scholar
  34. 34.
    Birkmeyer J, Finks J, O'Reilly A et al (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442PubMedCrossRefPubMedCentralGoogle Scholar
  35. 35.
    Beulens AJW, Brinkman WM, Van der Poel HG et al (2019) Linking surgical skills to postoperative outcomes: a Delphi study on the robot-assisted radical prostatectomy. J Robot Surg.  https://doi.org/10.1007/s11701-018-00916-9 CrossRefPubMedPubMedCentralGoogle Scholar
  36. 36.
    Goldenberg MG, Goldenberg L, Grantcharov TP (2017) Surgeon performance predicts early continence after robot-assisted radical prostatectomy. J Endourol 31(9):858–863PubMedCrossRefGoogle Scholar
  37. 37.
    Atug F, Sanli O, Duru AD (2018) Editorial comment on: utilizing machine learning and automated performance metrics to evaluate robot-assisted radical prostatectomy performance and predict outcomes by Hung et al. J Endourol 32(5):445PubMedCrossRefGoogle Scholar
  38. 38.
    Chen J, Remulla D, Nguyen JH et al (2019) Current status of artificial intelligence applications in urology and their potential to influence clinical practice. BJU Int.  https://doi.org/10.1111/bju.14852 CrossRefPubMedGoogle Scholar
  39. 39.
    Navaratnam A, Abdul-Muhsin H, Humphreys M (2018) Updates in urologic robot assisted surgery. F1000Res. https://doi.org/10.12688/f1000research.15480.1 CrossRefGoogle Scholar
  40. 40.
    Kong SH, Haouchine N, Soares R et al (2017) Robust augmented reality registration method for localization of solid organs’ tumors using CT-derived virtual biomechanical model and fluorescent fiducials. Surg Endosc 31(7):2863–2871PubMedCrossRefPubMedCentralGoogle Scholar
  41. 41.
    Bertolo R, Hung A, Porpiglia F et al (2019) Systematic review of augmented reality in urological interventions: the evidences of an impact on surgical outcomes are yet to come. World J Urol.  https://doi.org/10.1007/s00345-019-02711-z CrossRefPubMedPubMedCentralGoogle Scholar
  42. 42.
    van Oosterom MN, van der Poel HG, Navab N, van de Velde CJ, van Leeuwen FW (2018) Computer-assisted surgery: virtual-and augmented-reality displays for navigation during urological interventions. Curr Opin Urol 28(2):205–213PubMedCrossRefPubMedCentralGoogle Scholar
  43. 43.
    Pakhomov D, Premachandran V, Allan M, Azizian M, Navab N (2017) Deep residual learning for instrument segmentation in robotic surgery. arXiv preprint arXiv:1703.08580Google Scholar
  44. 44.
    Zhao Y, Guo S, Wang Y et al (2019) A CNN-based prototype method of unstructured surgical state perception and navigation for an endovascular surgery robot. Med Biol Eng Comput.  https://doi.org/10.1007/s11517-019-02002-0 CrossRefPubMedPubMedCentralGoogle Scholar
  45. 45.
    O’Sullivan S, Nevejans N, Allen C et al (2019) Legal, regulatory, and ethical frameworks for development of standards in artificial intelligence (AI) and autonomous robotic surgery. Int J Med Robot 15(1):e1968PubMedCrossRefPubMedCentralGoogle Scholar
  46. 46.
    Fichera L, Dillon NP, Zhang D et al (2017) Through the eustachian tube and beyond: new miniature robotic endoscope to see into the middle ear. IEEE Robot Autom Lett 2(3):1488–1494PubMedPubMedCentralCrossRefGoogle Scholar
  47. 47.
    Yang S, MacLachlan RA, Martel JN, Lobes LA Jr, Riviere CN (2016) Comparative evaluation of handheld robot-aided intraocular laser surgery. IEEE Trans Robot 32(1):246–251PubMedPubMedCentralCrossRefGoogle Scholar
  48. 48.
    Fornalik H, Fornalik N, Kincy T (2015) Advanced robotics: removal of a 25 cm pelvic mass. J Minim Invasive Gynecol 22(6S):S154PubMedCrossRefPubMedCentralGoogle Scholar
  49. 49.
    Tsai TY, Dimitriou D, Li JS, Kwon YM (2016) Does haptic robot-assisted total hip arthroplasty better restore native acetabular and femoral anatomy? Int J Med Robot 12(2):288–295PubMedCrossRefPubMedCentralGoogle Scholar
  50. 50.
    O’Sullivan S, Leonard S, Holzinger A et al (2019) Anatomy 101 for AI-driven robotics: explanatory, ethical and legal frameworks for development of cadaveric *skills training standards in autonomous robotic surgery/autopsy. Int J Med e2020Google Scholar
  51. 51.
    Chen CH, Suehn T, Illanes A et al (2018) Proximally placed signal acquision sensoric for robotic tissue tool interactions. Curr Dir Biomed Eng 4(1):67–70CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Iulia Andras
    • 1
    • 2
  • Elio Mazzone
    • 1
    • 3
    • 4
  • Fijs W. B. van Leeuwen
    • 1
    • 5
    • 6
  • Geert De Naeyer
    • 1
    • 3
  • Matthias N. van Oosterom
    • 5
    • 6
  • Sergi Beato
    • 1
  • Tessa Buckle
    • 5
  • Shane O’Sullivan
    • 7
  • Pim J. van Leeuwen
    • 6
  • Alexander Beulens
    • 8
    • 9
  • Nicolae Crisan
    • 2
  • Frederiek D’Hondt
    • 1
    • 3
  • Peter Schatteman
    • 1
    • 3
  • Henk van Der Poel
    • 6
  • Paolo Dell’Oglio
    • 1
    • 3
    • 5
    • 6
    Email author
  • Alexandre Mottrie
    • 1
    • 3
  1. 1.ORSI AcademyMelleBelgium
  2. 2.Department of UrologyIuliu Hatieganu University of Medicine and PharmacyCluj-NapocaRomania
  3. 3.Department of UrologyOnze Lieve Vrouw HospitalAalstBelgium
  4. 4.Department of Urology and Division of Experimental Oncology, URI, Urological Research InstituteIRCCS San Raffaele Scientific InstituteMilanItaly
  5. 5.Interventional Molecular Imaging Laboratory, Department of RadiologyLeiden University Medical CentreLeidenThe Netherlands
  6. 6.Department of Urology, Antoni Van Leeuwenhoek HospitalThe Netherlands Cancer InstituteAmsterdamThe Netherlands
  7. 7.Department of Pathology, Faculdade de MedicinaUniversidade de São PauloSão PauloBrazil
  8. 8.Department of UrologyCatharina HospitalEindhovenThe Netherlands
  9. 9.Netherlands Institute for Health Services (NIVEL)UtrechtThe Netherlands

Personalised recommendations