Skip to main content

Advertisement

SpringerLink
Log in

Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions

  • Review Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

An Erratum to this article was published on 11 January 2016

Abstract

Purpose

Advances in technology and computing play an increasingly important role in the evolution of modern surgical techniques and paradigms. This article reviews the current role of machine learning (ML) techniques in the context of surgery with a focus on surgical robotics (SR). Also, we provide a perspective on the future possibilities for enhancing the effectiveness of procedures by integrating ML in the operating room.

Methods

The review is focused on ML techniques directly applied to surgery, surgical robotics, surgical training and assessment. The widespread use of ML methods in diagnosis and medical image computing is beyond the scope of the review. Searches were performed on PubMed and IEEE Explore using combinations of keywords: ML, surgery, robotics, surgical and medical robotics, skill learning, skill analysis and learning to perceive.

Results

Studies making use of ML methods in the context of surgery are increasingly being reported. In particular, there is an increasing interest in using ML for developing tools to understand and model surgical skill and competence or to extract surgical workflow. Many researchers begin to integrate this understanding into the control of recent surgical robots and devices.

Conclusion

ML is an expanding field. It is popular as it allows efficient processing of vast amounts of data for interpreting and real-time decision making. Already widely used in imaging and diagnosis, it is believed that ML will also play an important role in surgery and interventional treatments. In particular, ML could become a game changer into the conception of cognitive surgical robots. Such robots endowed with cognitive skills would assist the surgical team also on a cognitive level, such as possibly lowering the mental load of the team. For example, ML could help extracting surgical skill, learned through demonstration by human experts, and could transfer this to robotic skills. Such intelligent surgical assistance would significantly surpass the state of the art in surgical robotics. Current devices possess no intelligence whatsoever and are merely advanced and expensive instruments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Troccaz J (2008) Medical robotics: where we come from, where we are and where we could go. Ind Robot Int J 35(4)

  2. Herron D, Marohn M (2008) A consensus document on robotic surgery. Surg Endosc 22(2):313–325

    Article  CAS  PubMed  Google Scholar 

  3. Camberlin C, Senn A, Leys M, De Laet C (2009) Robot-assisted surgery: health technology assessment. KCE reports 104A. Federaal Kenniscentrum voor de Gezondheidszorg

  4. ElSahwi KS, Hooper C, Leon MCD, Gallo TN, Ratner E, Silasi D-A, Santin AD, Schwartz PE, Rutherford TJ, Azodi M (2012) Comparison between 155 cases of robotic vs. 150 cases of open surgical staging for endometrial cancer. Gynecol Oncol 124(2):260–264

    Article  PubMed  Google Scholar 

  5. Brandao LF, Autorino R, Laydner H, Haber G-P, Ouzaid I, Sio MD, Perdonà S, Stein RJ, Porpiglia F, Kaouk JH (2014) Robotic versus laparoscopic adrenalectomy: a systematic review and meta-analysis. Eur Urol 65(6):1154–1161

    Article  PubMed  Google Scholar 

  6. Morino M, Pellegrino L, Giaccone C, Garrone C, Rebecchi F (2006) Randomized clinical trial of robot-assisted versus laparoscopic nissen fundoplication. Br J Surg 93(5):553–558

    Article  CAS  PubMed  Google Scholar 

  7. Nakadi I, Mélot C, Closset J, Moor V, Bétroune K, Feron P, Lingier P, Gelin M (2006) Evaluation of da Vinci nissen fundoplication clinical results and cost minimization. World J Surg 30(6):1050–1054

    Article  PubMed  Google Scholar 

  8. Park JS, Choi G-S, Park SY, Kim HJ, Ryuk JP (2012) Randomized clinical trial of robot-assisted versus standard laparoscopic right colectomy. Br J Surg 99(9):1219–1226

    Article  CAS  PubMed  Google Scholar 

  9. van den Berg J, Miller S, Duckworth D, Hu H, Wan A, Fu X, Goldberg K, Abbeel P (2010) Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 2074–2081

  10. Kwoh Y, Hou J, Jonckheere E, Hayati S (1988) A robot with improved absolute positioning accuracy for CT guided stereotactic brain surgery. IEEE Trans Biomed Eng 35(2):153–160

    Article  CAS  PubMed  Google Scholar 

  11. Paul H, Bargar W, Mittlestadt B, Musits B, Taylor R, Kazanzides P, Zuhars J, Williamson B, Hanson W (1992) Development of a surgical robot for cementless total hip arthroplasty. Clin Orthop Relat Res 285:57–66

    PubMed  Google Scholar 

  12. Burckhardt CW, Flury P, Glauser D (1995) Stereotactic brain surgery. Eng Med Biol Mag IEEE 14(3):314–317

    Article  Google Scholar 

  13. Adler J, Chang S, Murphy M, Doty J, Geis P, Hancock S (1997) The Cyberknife: a frameless robotic system for radiosurgery. Stereotact Funct Neurosurg 69(1–4):124–128

    Article  PubMed  Google Scholar 

  14. Kang H (2002) Robotic assisted suturing in minimally invasive surgery. Ph.D. thesis, Rensselaer Polytechnic Institute , Troy, New York

  15. Jackson R, Cavusoglu M (2013) Needle path planning for autonomous robotic surgical suturing. In: IEEE international conference on robotics and automation (ICRA), pp 1669–1675

  16. Nageotte F, Zanne P, Doignon C, de Mathelin M (2009) Stitching planning in laparoscopic surgery: towards robot-assisted suturing. Int J Robot Res 28(10):1303–1321

    Article  Google Scholar 

  17. Jansen R, Hauser K, Chentanez N, van der Stappen F, Goldberg K (2009) Surgical retraction of non-uniform deformable layers of tissue: 2D robot grasping and path planning. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 4092–4097

  18. Patil S, Alterovitz R (May 2010) Toward automated tissue retraction in robot-assisted surgery. In: IEEE international conference on robotics and automation (ICRA), pp 2088–2094

  19. Muradore R, Bresolin D, Geretti L, Fiorini P, Villa T (2011) Robotic surgery—formal verification of plans. Robot Autom Mag IEEE 18(3):24–32

    Article  Google Scholar 

  20. Brett, P, Taylor R, Proops D, Coulson C, Reid A, Griffiths M (2007) A surgical robot for cochleostomy. In: Proceedings of 29th annual international conference of the IEEE engineering in medicine and biology society (EMBS), pp 1229–1232

  21. Larsson J, Hayes-Roth B (1998) Guardian: intelligent autonomous agent for medical monitoring and diagnosis. Intell Syst Their Appl IEEE 13(1):58–64

    Article  Google Scholar 

  22. Frei C, Gentilini A, Derighetti M, Glattfelder A, Morari M, Schnider T, Zbinden A (1999) Automation in anesthesia. In: Proceedings of the American control conference (ACC), pp 1258–1263

  23. Mitchell TM (1997) Machine learning. McGraw-Hill, New York

    Google Scholar 

  24. Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin

    Google Scholar 

  25. Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292

    Google Scholar 

  26. Price B, Boutilier C (2003) A Bayesian approach to imitation in reinforcement learning. In: Proceedings of the 18th international joint conference on artificial intelligence (IJCAI), pp 712–720

  27. Birkhimer CE (2005) Extracting human strategies for use in robotic assembly. Ph.D. thesis, Case Western Reserve University, Cleveland, OH, USA. AAI3159443

  28. Hannaford B, Lee P (1991) Hidden Markov model analysis of force/torque information in telemanipulation. Int J Robot Res 10(5):528–539

    Article  Google Scholar 

  29. Itabashi K, Hayakawa K, Suzuki T, Okuma S, Fujiwara F (1997) Modeling of the peg-in-hole task based on impedance parameters and HMM. Proc IEEE/RSJ Int Conf Intell Robots Syst 1:451–457

    Google Scholar 

  30. Asada H, Liu S (1991) Transfer of human skills to neural net robot controllers. Proc IEEE Int Conf Robot Autom 3:2442–2448

    Google Scholar 

  31. Kaiser M, Dillmann R (1996) Building elementary robot skills from human demonstration. Proc IEEE Int Conf Robot Autom 3:2700–2705

    Article  Google Scholar 

  32. Myers D (1999) An approach to automated programming of industrial robots. Proc IEEE Int Conf Robot Autom 4:3109–3114

    Article  Google Scholar 

  33. Skubic M, Volz R (2000) Acquiring robust, force-based assembly skills from human demonstration. IEEE Trans Robot Autom 16(6):772–781

    Article  Google Scholar 

  34. Ng AY, Russell SJ (2000) Algorithms for inverse reinforcement learning. In: Proceedings of the international conference on machine learning (ICML), pp 663–670

  35. Abbeel P, Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the international conference on machine learning (ICML)

  36. Ramachandran D, Amir E (2007) Bayesian inverse reinforcement learning. In: Proceedings of the international joint conference on artificial intelligence (IJCAI), pp 2586–2591

  37. Kirk DE (1970) Optimal control theory: an introduction. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  38. Reznick RK, MacRae H (2006) Teaching surgical skills—changes in the wind. N Eng J Med 355(25):2664–2669

    Article  CAS  Google Scholar 

  39. Jog A, Itkowitz B, Liu M, DiMaio S, Hager G, Curet M, Kumar R (2011) Towards integrating task information in skills assessment for dexterous tasks in surgery and simulation. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 5273–5278. IEEE

  40. Kenney PA, Wszolek MF, Gould JJ, Libertino JA, Moinzadeh A (2009) Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology 73(6):1288–1292

    Article  PubMed  Google Scholar 

  41. Tedesco MM, Pak JJ, Harris EJ Jr, Krummel TM, Dalman RL, Lee JT (2008) Simulation-based endovascular skills assessment: the future of credentialing? J Vasc Surg 47(5):1008–1014

    Article  PubMed  Google Scholar 

  42. Reiley CE, Lin HC, Yuh DD, Hager GD (2011) Review of methods for objective surgical skill evaluation. Surg Endosc 25(2):356–366

    Article  PubMed  Google Scholar 

  43. Van Hove PD, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987

    Article  PubMed  Google Scholar 

  44. Niitsu H, Hirabayashi N, Yoshimitsu M, Mimura T, Taomoto J, Sugiyama Y, Murakami S, Saeki S, Mukaida H, Takiyama W (2013) Using the objective structured assessment of technical skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg Today 43(3):271–275

    Article  PubMed  PubMed Central  Google Scholar 

  45. Bridgewater B, Grayson AD, Jackson M, Brooks N, Grotte GJ, Keenan DJM, Millner R, Fabri BM, Mark J (2003) Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. Br Med J 327(7405):13–17

    Article  Google Scholar 

  46. Dosis A, Aggarwal R, Bello F, Moorthy K, Munz Y, Gillies D, Darzi A (2005) Synchronized video and motion analysis for the assessment of procedures in the operating theater. Arch Surg 140(3):293–299

    Article  PubMed  Google Scholar 

  47. Schijven MP, Jakimowicz J, Schot C (2002) The advanced dundee endoscopic psychomotor tester (ADEPT) objectifying subjective psychomotor test performance. Surg Endosc 16(6):943–948

    Article  CAS  PubMed  Google Scholar 

  48. Botden SMBI, Buzink SN, Schijven MP, Jakimowicz JJ (2008) ProMIS augmented reality training of laparoscopic procedures face validity. J Soc Simul Healthc 3(2):97–102

    Article  Google Scholar 

  49. Hattori M, Egi H, Tokunaga M, Suzuki T, Ohdan H, Kawahara T (July 2012) The integrated deviation in the HUESAD (Hiroshima University Endoscopic Surgical Assessment Device) represents the surgeon’s visual-spatial ability. In: Proceedings of The 2012 ICME international conference on complex medical engineering (CME), pp 316–320

  50. van Empel PJ, van Rijssen LB, Commandeur JP, Verdam MGE, Huirne JA, Scheele F, Bonjer HJ, Meijerink WJ (2012) Validation of a new box trainer-related tracking device: the TrEndo. Surg Endosc 26:2346–2352

    Article  PubMed  PubMed Central  Google Scholar 

  51. DiMaio SP, Hasser CJ (2008) The da Vinci research interface. In: MICCAI workshop on systems and architecture for computer assisted interventions, MIDAS Journal

  52. Kazanzides P, DiMaio S, Deguet A, Vagvolgyi B, Balicki M, Schneider C, Kumar R, Jog A, Itkowitz B, Hasser C, Taylor R (2010) The surgical assistant workstation (SAW) in minimally-invasive surgery and microsurgery. In: Proceedings of MICCAI workshop on systems and architecture for computer assisted interventions (SACAI)

  53. Rosen J, Brown JD, Chang L, Sinanan MN, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Trans Biomed Eng 53(3):399–413

    Article  PubMed  Google Scholar 

  54. Speidel S, Zentek T, Sudra G, Gehrig T, Müller-Stich BP, Gutt C, Dillmann R (2009) Recognition of surgical skills using hidden markov models. In: Proceedings of SPIE medical imaging, vol 7261, p 25

  55. Lin HC, Shafran I, Murphy TE, Okamura AM, Yuh DD, Hager GD (2005) Automatic detection and segmentation of robot-assisted surgical motions. In: Proceedings of medical image computing and computer-assisted intervention (MICCAI), Springer, pp 802–810

  56. Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB (2011) Observational tools for assessment of procedural skills: a systematic review. Am J Surg 202(4):469–480

    Article  PubMed  Google Scholar 

  57. Shah CV, Riga CV, Stoyanov D, Yang G-Z, Cheshire NJW, Bicknell CD (2011) Video motion analysis for approaching objective assessment of catheter-based endovascular interventions. In: Proceedings of The Hamlyn symposium on medical robotics, pp 93–94

  58. Neequaye SK, Aggarwal R, Van Herzeele I, Darzi A, Cheshire NJ (2007) Endovascular skills training and assessment. J Vasc Surg 46(5):1055–1064

    Article  PubMed  Google Scholar 

  59. Reiley CE, Hager GD (2009) Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In: Proceedings of medical image computing and computer-assisted intervention (MICCAI), Springer, pp 435–442

  60. Lin H, Shafran I, Yuh D, Hager G (2006) Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg 11(5):220–230

    Article  PubMed  Google Scholar 

  61. Reiley C, Lin H, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh D, Hager (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inf 132:396–401

    Google Scholar 

  62. Rafii-Tari H, Liu J, Lee S-L, Bicknell C, Yang G-Z (2013) Learning-based modeling of endovascular navigation for collaborative robotic catheterization. In: Proceedings of medical image computing and computer-assisted intervention (MICCAI), vol 8150, Springer, pp 369–377

  63. Padoy N, Blum T, Essa I, Feussner H, Berger M-O, Navab N (2007) A boosted segmentation method for surgical workflow analysis. In: Medical image computing and computer-assisted intervention (MICCAI), vol 4791. Lecture Notes in Computer Science, Springer, Berlin Heidelberg, pp 102–109

  64. Ahmadi S-A, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2006) Recovery of surgical workflow without explicit models. In: Medical image computing and computer-assisted intervention (MICCAI), vol 4190. Springer, pp 420–428

  65. Reiley C, Hager G (2009) Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In: Medical image computing and computer-assisted intervention (MICCAI), vol 5761. Springer, pp 435–442

  66. Varadarajan B, Reiley C, Lin H, Khudanpur S, Hager G (2009) Data-derived models for segmentation with application to surgical assessment and training. In: Medical image computing and computer-assisted intervention (MICCAI), vol 5761, Springer, pp 426–434

  67. Varadarajan B (2011) Learning and inference algorithms for dynamical system models of dextrous motion. Ph.D. thesis, The Johns Hopkins University

  68. Tao L, Elhamifar E, Khudanpur S, Hager GD, Vidal R (2012) Sparse hidden markov models for surgical gesture classification and skill evaluation. In: Proceedings of information processing in computer-assisted interventions (IPCAI), Springer, pp 167–177

  69. Rosen J, Brown J, Chang L, Sinanan M, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete markov model. IEEE Trans Biomed Eng 53(3):399–413

    Article  PubMed  Google Scholar 

  70. Moustris GP, Hiridis SC, Deliparaschos KM, Konstantinidis KM (2011) Evolution of autonomous and semi-autonomous robotic surgical systems: a review of the literature. Int J Med Robot Comput Assist Surg 7(4):375–392

    Article  CAS  Google Scholar 

  71. Drake J, Joy M, Goldenberg A, Kreindler D (1991) Computer and robotic assisted resection of brain tumours. In: Proceedings of the fifth international conference on advanced robotics (ICAR), ’Robots in Unstructured Environments’, vol 1, pp 888–892

  72. De Momi E, Ferrigno G (2010) Robotic and artificial intelligence for keyhole neurosurgery: the robocast project, a multi-modal autonomous path planner. Proc Inst Mech Eng H 224(5):715–727

    Article  PubMed  Google Scholar 

  73. Benabid A, Cinquin P, Lavalle S, Bas JL, Demongeot J, de Rougemont J (1987) Computer-driven robot for stereotactic surgery connected to CT scan and magnetic resonance imaging. Technological design and preliminary results. Stereotact Funct Neurosurg 50(1–6):153–154

    Article  CAS  Google Scholar 

  74. Taylor R, Paul H, Mittelstadt B, Hanson W, Kazanzides P, Zuhars J, Glassman E, Musits B, Williamson W, Bargar W (1990) An image-directed robotic system for precise orthopaedic surgery. In: Proceedings of the 12th annual international conference of the IEEE on engineering in medicine and biology society (EMBC), pp 1928–1930

  75. Taylor RH, Mittelstadt BD, Paul HA, Hanson W, Kazanzides P, Zuhars JF, Williamson B, Musits BL, Glassman E, Bargar WL (1994) An image-directed robotics system for precise orthopedic surgery. IEEE Trans Robot Autom 10(3):261–275

    Article  Google Scholar 

  76. Kazanzides P, Mittelstadt B, Musits B, Bargar W, Zuhars J, Williamson B, Cain P, Carbone E (1995) An integrated system for cementless hip replacement. Eng Med Biol Mag IEEE 14(3):307–313

    Article  Google Scholar 

  77. Fu L, Du Z, Sun L (2004) A novel robot-assisted bonesetting system. Proc IEEE/RSJ Int Conf Intell Robots Syst 3:2247–2252

    Google Scholar 

  78. Mönnich H, Stein D, Raczkowsky J, Worn H (May 2010) An automatic and complete self-calibration method for robotic guided laser ablation. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 1086–1087

  79. Schorr O, Münchenberg J, Raczkowsky J, Wörn H (2001) KasOp—a generic system for pre- and intraoperative surgical assistance and guidance. In: Proceedings of the 15th international congress and exhibition of computer assisted radiology and surgery (CARS)

  80. Engel D, Raczkowsky J, Wörn H (2001) A safe robot system for craniofacial surgery. In: Proceedings IEEE international conference on robotics and automation (ICRA), vol 2, pp 2020–2024

  81. Coulson CJ, Reid AP, Proops DW (2008) A cochlear implantation robot in surgical practice. In: Proceedings of the 15th international conference on mechatronics and machine vision in practice (M2VIP), pp 173–176

  82. Weber S, Belle B, Brett P, Du X, Caversaccio M, Proops D, Coulson C, Reid A (2013) Minimally invasive, robot assisted cochlear implantation. In: Proceedings of the 3rd joint workshop on new technologies for computer/robot assisted surgery (CRAS), pp 134–137

  83. Coulson C, Taylor R, Reid A, Griffiths M, Proops D, Brett P (2008) An autonomous surgical robot for drilling a cochleostomy: preliminary porcine trial. Clin Otolaryngol 33(4):343–347

    Article  CAS  PubMed  Google Scholar 

  84. Adler J, Murphy M, Martin J, Chang S, Hancock S (1999) Image-guided robotic radiosurgery. Neurosurgery 44(6):1299–1306

    PubMed  Google Scholar 

  85. Ernst F, Schweikard A (2009) Forecasting respiratory motion with accurate online support vector regression (SVRpred). Int J Comput Assist Radiol Surg 4(5):439–447

    Article  PubMed  Google Scholar 

  86. Kassahun Y, Yu B, Vander Poorten E (2013) Learning catheter-aorta interaction model using joint probability densities. In: Proceedings of the 3rd joint workshop on new technologies for computer/robot assisted surgery, pp 158–160

  87. Bauernschmitt R, Schirmbeck EU, Knoll A, Mayer H, Nagy I, Wessel N, Wildhirt SM, Lange R (2005) Towards robotic heart surgery: introduction of autonomous procedures into an experimental surgical telemanipulator system. Int J Med Robot Comput Assist Surg 1(3):74–79

    Article  CAS  Google Scholar 

  88. Ji C, Hou Z-G, Xie X-L (2011) An image-based guidewire navigation method for robot-assisted intravascular interventions. In: Engineering in medicine and biology society, EMBC, 2011 annual international conference of the IEEE, pp 6680–6685. IEEE

  89. Tercero C, Ikeda S, Uchiyama T, Fukuda T, Arai F, Ono Y (2006) Catheter insertion mechanism and feedback control using magnetic motion capture sensor. In: Proceedings of international joint conference on SICE-ICASE, pp 1856–1859

  90. Tercero Villagran C, Ikeda S, Fukuda T, Sekiyama K, Okada Y, Uchiyama T, Negoro M, Takahashi I (2007) Catheter insertion path reconstruction with autonomous system for endovascular surgery. In: Proceedings of international symposium on computational intelligence in robotics and automation (CIRA), pp 398–403. IEEE

  91. Tercero C, Ikeda S, Uchiyama T, Fukuda T, Arai F, Okada Y, Ono Y, Hattori R, Yamamoto T, Negoro M, Takahashi I (2007) Autonomous catheter insertion system using magnetic motion capture sensor for endovascular surgery. Int J Med Robot Comput Assist Surg 3:52–58

    Article  CAS  Google Scholar 

  92. Harris S, Arambula-Cosio F, Mei Q, Hibberd R, Davies B, Wickham J, Nathan M, Kundu B (1997) The probot—an active robot for prostate resection. Proc Inst Mech Eng Part H J Eng Med 211(4):317–325

    Article  CAS  Google Scholar 

  93. Muntener M, Patriciu A, Petrisor D, Mazilu D, Bagga H, Kavoussi L, Cleary K, Stoianovici D (2006) Magnetic resonance imaging compatible robotic system for fully automated brachytherapy seed placement. Urology 68(6):1313–1317

    Article  PubMed  PubMed Central  Google Scholar 

  94. Patriciu A, Petrisor D, Muntener M, Mazilu D, Schar M, Stoianovici D (2007) Automatic brachytherapy seed placement under MRI guidance. IEEE Trans Biomed Eng 54(8):1499–1506

    Article  PubMed  PubMed Central  Google Scholar 

  95. Davies B, Hibberd R, Coptcoat M (1989) A surgeon robot for prostatectomy—a laboratory evaluation. J Med Eng Technol 13:273–277

    Article  CAS  PubMed  Google Scholar 

  96. Casals A, Amat J, Laporte E (1996) Automatic guidance of an assistant robot in laparoscopic surgery. Proc IEEE Int Conf Robot Autom 1:895–900

    Article  Google Scholar 

  97. Wei G-Q, Arbter K, Hirzinger G (1997) Real-time visual servoing for laparoscopic surgery. Controlling robot motion with color image segmentation. Eng Med Biol Mag IEEE 16(1):40–45

    Article  CAS  Google Scholar 

  98. Krupa A, Doignon C, Gangloff J, Mathelin M, Solert L, Morel G (2001) Towards semi-autonomy in laparoscopic surgery through vision and force feedback control. In: Rus D, Singh S (eds) Experimental Robotics VII, vol 271, Lecture Notes in Control and Information Sciences. Springer, Berlin Heidelberg, pp 189–198

  99. Krupa A, Gangloff J, Doignon C, de Mathelin M, Morel G, Leroy J, Soler L, Marescaux J (2003) Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE Trans Robot Autom 19(5):842–853

    Article  Google Scholar 

  100. Weede O, Mönnich H, Müller B, Wörn H (2011) An intelligent and autonomous endoscopic guidance system for minimally invasive surgery. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 5762–5768

  101. Krupa A, Gangloff J, de Mathelin M, Doignon C, Morel G, Soler L, Leroy J, Marescaux J (2002) Autonomous retrieval and positioning of surgical instruments in robotized laparoscopic surgery using visual servoing and laser pointers. In: Proceedings of IEEE international conference on robotics and automation (ICRA), vol 4, pp 3769–3774

  102. Taguchi K, Ohnishi K (2008) A bilateral teleoperation method using an autonomous control based on DFT modal decomposition. In: Proceedings of the 34th annual conference of IEEE on industrial electronics (IECON), pp 2497–2502. IEEE

  103. Estebanez B, Bauzano E, Muoz V (2011) Surgical tools pose estimation for a multimodal hmi of a surgical robotic assistant. In: 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 2121–2126

  104. Ko S-Y, Kwon D-S (2004) A surgical knowledge based interaction method for a laparoscopic assistant robot. In: Proceedings of 13th IEEE international workshop on robot and human interactive communication (ROMAN), pp 313–318. IEEE

  105. Gangloff J, Ginhoux R, De Mathelin M, Soler L, Marescaux J (2006) Model predictive control for compensation of cyclic organ motions in teleoperated laparoscopic surgery. IEEE Trans Control Syst Technol 14(2):235–246

    Article  Google Scholar 

  106. Bauzano E, Mu andoz V, Garcia-Morales I (2010) Auto-guided movements on minimally invasive surgery for surgeon assistance. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 1843–1848. IEEE

  107. Osa T, Staub C, Knoll A (2010) Framework of automatic robot surgery system using visual servoing. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 1837–1842

  108. Staub C, Knoll A, Osa T, Bauernschmitt R (March 2010) Autonomous high precision positioning of surgical instruments in robot-assisted minimally invasive surgery under visual guidance. In: Proceedings of sixth international conference on autonomic and autonomous systems (ICAS), pp 64–69

  109. Dumpert J, Lehman A, Wood N, Oleynikov D, Farritor S (2009) Semi-autonomous surgical tasks using a miniature in vivo surgical robot. In: Annual international conference of IEEE on engineering in medicine and biology society (EMBC), pp 266–269. IEEE

  110. King B, Reisner L, Pandya A, Composto A, Ellis R, Klein M (2013) Towards an autonomous robot for camera control during laparoscopy surgery. J Laparoendosc Adv Surg Tech 23(12):1027–1030

    Article  Google Scholar 

  111. Bauzano E, Garcia-Morales I, del Saz-Orozco P, Fraile J, Muoz V (2013) A minimally invasive surgery robotic assistant for HALS–SILS techniques. Comput Methods Prog Biomed 112(2):272–283

    Article  CAS  Google Scholar 

  112. Cornella J, Elle O-J, Ali W, Samset E (2008) Intraoperative navigation of an optically tracked surgical robot. Med Image Comput Assist Interv 11(2):587–594

    Google Scholar 

  113. Chen G, Pham MT, Redarce T (2008) A semi-autonomous micro-robotic system for colonoscopy. In: Proceedings of IEEE international conference on robotics and biomimetics (ROBIO), pp 703–708. IEEE

  114. Kang H, Wen J (2000) Autonomous suturing using minimally invasive surgical robots. In: Proceedings of IEEE international conference on control applications (CCA), pp 742–747

  115. Mayer H, Gomez F, Wierstra D, Nagy I, Knoll A, Schmidhuber J (2006) A system for robotic heart surgery that learns to Tie Knots using recurrent neural networks. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 543–548

  116. Hynes P, Dodds G, Wilkinson AJ (2006) Uncalibrated visual-servoing of a dual-arm robot for MIS Suturing. In: Proceedings of the first IEEE/RAS-EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 420–425

  117. Mayer HG, Nagy I, Knoll A, Schirmbeck EU, Bauernschmitt R (2004) The EndoPAR system for minimally invasive robotic surgery. Proc IEEE/RSJ Int Conf Intell Robots Syst 4:3637–3642

    Google Scholar 

  118. Mayer H, Wierstra D, Gomez F, Nagy I, Knoll A, Schmidhuber J (2008) A system for robotic heart surgery that learns to tie knots using recurrent neural networks. Adv Robot 22(13–14):1521–1537

    Article  Google Scholar 

  119. Mayer H, Nagy I, Burschka D, Knoll A, Braum E, Lange R, Bauernschmitt R (2008) Automation of manual tasks for minimally invasive surgery. In: Proceedings of international conference on autonomic and autonomous systems (ICAS), pp 260–265

  120. Chatelain P, Krupa A, Marchal M (2013) Real-time needle detection and tracking using a visually servoed 3D ultrasound probe. In: IEEE international conference on robotics and automation (ICRA), pp 1676–1681. IEEE

  121. Mayer H, Nagy I, Knoll A, Braun E, Lange R, Bauernschmitt R (2007) Adaptive control for human-robot skilltransfer: trajectory planning based on fluid dynamics. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 1800–1807. IEEE

  122. Reiley C, Plaku E, Hager G (2010) Motion generation of robotic surgical tasks: Learning from expert demonstrations. In: Proceedings of annual international conference of the IEEE on engineering in medicine and biology society (EMBC), pp 967–970. IEEE

  123. Chow D-L, Newman W (2013) Improved knot-tying methods for autonomous robot surgery. In: Proceedings of IEEE international conference on automation science and engineering (CASE), pp 461–465. IEEE

  124. Kang H, Wen J (2001) Endobot: a robotic assistant in minimally invasive surgeries. In: IEEE international conference on robotics and automation, 2001. Proceedings 2001 ICRA, vol 2, pp 2031–2036. IEEE

  125. Schulman J, Ho J, Lee C, Abbeel P (2013) Learning from Demonstrations through the use of non-rigid registration. In: Proceedings of the 16th international symposium on robotics research (ISRR)

  126. Kang H, Wen J (2001) Robotic assistants aid surgeons during minimally invasive procedures. Eng Med Biol Mag IEEE 20(1):94–104

    Article  CAS  Google Scholar 

  127. Kantor GS (2015) Intravenous catheter complications. http://www.netwellness.org/healthtopics/anesthesiology/ivcomplications.cfm. [Online; Accessed 16 Sep 2015]

  128. Zong G, Hu Y, Li D, Sun X (2006) Visually servoed suturing for robotic microsurgical keratoplasty. In: 2006 IEEE/RSJ international conference on intelligent robots and systems, pp 2358–2363. IEEE

  129. Mallapragada V, Sarkar N, Podder T (2008) Autonomous coordination of imaging and tumor manipulation for robot assisted breast biopsy. In: Proceedings of 2nd IEEE RAS EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 676–681. IEEE

  130. Mallapragada V, Sarkar N, Podder T (2011) Toward a robot-assisted breast intervention system. IEEE/ASME Trans Mechatron 16(6):1011–1020

    Article  Google Scholar 

  131. Liang K, Light E, Rogers A, Von Allmen D, Smith S (2009) 3-D ultrasound guidance of autonomous surgical robotics: feasibility studies. In: Proceedings of IEEE international ultrasonics symposium (IUS), pp 582–585. IEEE

  132. Liang K, Rogers A, Light E, von Allmen D, Smith S (2010) Three-dimensional ultrasound guidance of autonomous robotic breast biopsy: feasibility study. Ultrasound Med Biol 36(1):173–177

    Article  PubMed  PubMed Central  Google Scholar 

  133. Bonfe M, Preda N, Secchi C, Ferraguti F, Muradore R, Repele L, Lorenzi G, Gasperotti L, Fiorini P (2013) Automated surgical task execution: the needle insertion case. In: Proceedings of 3rd joint workshop on new technologies for computer/robot assisted surgery (CRAS), pp 45–47

  134. Whitman J, Fronheiser MP, Ivancevich NM, Smith SW (2007) Autonomous surgical robotics using 3-d ultrasound guidance: feasibility study. Ultrason Imag 29(4):213–219

    Article  Google Scholar 

  135. Staub C, Osa T, Knoll A, Bauernschmitt R (2010) Automation of tissue piercing using circular needles and vision guidance for computer aided laparoscopic surgery. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 4585–4590

  136. Zivanovic A, Davies BL (2000) A robotic system for blood sampling. Inf Technol Biomed IEEE Trans 4(1):8–14

    Article  CAS  Google Scholar 

  137. Osa T, Haniu T, Harada K, Sugita N, Mitsuishi M (2013) Perforation risk detector using demonstration-based learning for teleoperated robotic surgery. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 2572–2577. IEEE

  138. Nichols K, Okamura A (2013) Autonomous robotic palpation: Machine learning techniques to identify hard inclusions in soft tissues. In: 2013 IEEE International Conference on robotics and automation (ICRA), pp 4384–4389. IEEE

  139. Abolmaesumi P, Salcudean S, Zhu W-H, Sirouspour M, DiMaio S (2002) Image-guided control of a robot for medical ultrasound. IEEE Trans Robot Autom 18(1):11–23

    Article  Google Scholar 

  140. Li T, Kermorgant O, Krupa A (2012) Maintaining visibility constraints during tele-echography with ultrasound visual servoing. In: IEEE international conference on robotics and automation (ICRA), pp 4856–4861. IEEE

  141. Gumprecht JDJ, Baumann M, Menz A, Stolzenburg J-U, Lueth T (2012) A modular control concept for a flat-panel ultrasound robot. In: 2012 4th IEEE RAS EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 907–912. IEEE

  142. Mylonas GP, Giataganas P, Chaudery M, Vitiello V, Darzi A, Yang G-Z (2013) Autonomous eFAST ultrasound scanning by a robotic manipulator using learning from demonstrations. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 3251–3256. IEEE

  143. Giataganas P, Vitiello V, Simaiaki V, Lopez E, Yang G-Z (2013) Cooperative in situ microscopic scanning and simultaneous tissue surface reconstruction using a compliant robotic manipulator. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 5378–5383. IEEE

  144. Fichera L, Pardo D, Mattos LS (2013) Modeling tissue temperature dynamics during laser exposure. In: Proceedings of the international work conference on artificial neural networks (IWANN), pp 96–106

  145. Kehoe B, Kahn G, Mahler J, Kim J, Lee A, Lee A, Nakagawa K, Patil S, Boyd WD, Abbeel P, Goldberg K (2014) Autonomous multilateral debridement with the raven surgical robot. In: Proceedings of the IEEE international conference on robotics and automation (ICRA)

  146. Garcia P, Rosen J, Kapoor C, Noakes M, Elbert G, Treat M, Ganous T, Hanson M, Manak J, Hasser C, Rohler D, Satava R (2009) Trauma pod: a semi-automated telerobotic surgical system. Int J Med Robot Comput Assist Surg 5(2):136–146

    Article  Google Scholar 

  147. Treat M, Amory S, Downey P, Taliaferro D (2006) Initial clinical experience with a partly autonomous robotic surgical instrument server. Surg Endosc Other Interv Tech 20(8):1310–1314

    Article  CAS  Google Scholar 

  148. Mönnich H, Botturi D, Raczko Raczkowsky H, Wörn H (2009) System architecture for workflow controlled robotic surgery. J Inf Technol Healthc 6

  149. Bonfe M, Boriero F, Dodi R, Fiorini P, Morandi A, Muradore R, Pasquale L, Sanna A, Secchi C (2012) Towards automated surgical robotics: a requirements engineering approach. In: Proceedings of the 4th IEEE RAS EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 56–61

  150. Bresolin D, Di Guglielmo L, Geretti L, Muradore R, Fiorini P, Villa T (2012) Open problems in verification and refinement of autonomous robotic systems. In: Proceedings of the 15th euromicro conference on digital system design (DSD), pp 469–476. IEEE

  151. Muradore R, Zerbato D, Vezzaro L, Gasperotti L, Fiorini P (2012) From simulation to abstract modeling of surgical operations. In: Proceedings of the 2nd joint workshop on new technologies for computer/robot assisted surgery (CRAS), Madrid

  152. Botturi D, Fiorini P (2005) Optimal control for autonomous task execution. In: 44th IEEE conference on decision and control, 2005 and 2005 European control conference. CDC-ECC ’05, pp 3525–3530. IEEE

  153. Monnich H, Worn H, Stein D (2012) OP sense—a robotic research platform for telemanipulated and automatic computer assisted surgery. In: Proceedings of 12th IEEE international workshop on advanced motion control (AMC), pp 1–6. IEEE

  154. Ginhoux R, Gangloff J, De Mathelin M, Soler L, Sanchez M, Marescaux J (2005) Active filtering of physiological motion in robotized surgery using predictive control. IEEE Trans Robot 21(1):67–79

    Article  Google Scholar 

  155. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  CAS  PubMed  Google Scholar 

  156. Calinon S, Bruno D, Malekzadeh MS, Nanayakkara T, Caldwell DG (2014) Human-robot skills transfer interfaces for a flexible surgical robot. Comput Methods Prog Biomed 116(2):81–96

    Article  Google Scholar 

  157. Maron JK, Zenati MA (2013) Framework for managing cognitive load in a data-rich robotic operating room. In: Proceedings of the 3rd joint workshop on new technologies for computer/robot assisted surgery (CRAS), pp 150–153

  158. Endsley MR (1995) Toward a theory of situation awareness in dynamic system. Hum Factors J Hum Factors Ergon Soc 37(1):32–64

    Article  Google Scholar 

  159. Jacob MG, Li Y-T, Akingba GA, Wachs JP (2013) Collaboration with a robotic scrub nurse. Commun ACM 56(5):68–75

    Article  Google Scholar 

  160. Meyfroidt G, Güizab F, Ramonb J, Bruynooghe M (2009) Machine learning techniques to examine large patient databases. Best Pract Res Clin Anaesthesiol 23(1):127–143

    Article  PubMed  Google Scholar 

  161. Petersen J, Rodriguez Baena F (Nov 2013) A dynamic active constraints approach for hands-on robotic surgery. In: 2013 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 1966–1971

Download references

Funding

This research has been funded by the European Commission’s 7th Framework Programme FP7-ICT, by the project CASCADE under Grant Agreement No.601021.

Author information

Authors and Affiliations

  1. Robotics Innovation Center, German Research Center for Artificial Intelligence, Robert-Hooke-Str. 1, 28359, Bremen, Germany

    Yohannes Kassahun

  2. Faculty 3 - Mathematics and Computer Science, University of Bremen, Robert-Hooke-Str. 1, 28359, Bremen, Germany

    Bingbin Yu, Abraham Temesgen Tibebu & Jan Hendrik Metzen

  3. Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK

    Danail Stoyanov

  4. Hamlyn Centre for Robotic Surgery, Imperial College London, London, UK

    Stamatia Giannarou

  5. Department of Mechanical Engineering, University of Leuven, Celestijnenlaan 300B, 3001, Heverlee, Belgium

    Emmanuel Vander Poorten

Authors
  1. Yohannes Kassahun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bingbin Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abraham Temesgen Tibebu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Danail Stoyanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stamatia Giannarou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jan Hendrik Metzen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Emmanuel Vander Poorten
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yohannes Kassahun.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kassahun, Y., Yu, B., Tibebu, A.T. et al. Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int J CARS 11, 553–568 (2016). https://doi.org/10.1007/s11548-015-1305-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-015-1305-z

Keywords

Search

Navigation