Advertisement

Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions

  • Yohannes KassahunEmail author
  • Bingbin Yu
  • Abraham Temesgen Tibebu
  • Danail Stoyanov
  • Stamatia Giannarou
  • Jan Hendrik Metzen
  • Emmanuel Vander Poorten
Review Article

Abstract

Purpose

Advances in technology and computing play an increasingly important role in the evolution of modern surgical techniques and paradigms. This article reviews the current role of machine learning (ML) techniques in the context of surgery with a focus on surgical robotics (SR). Also, we provide a perspective on the future possibilities for enhancing the effectiveness of procedures by integrating ML in the operating room.

Methods

The review is focused on ML techniques directly applied to surgery, surgical robotics, surgical training and assessment. The widespread use of ML methods in diagnosis and medical image computing is beyond the scope of the review. Searches were performed on PubMed and IEEE Explore using combinations of keywords: ML, surgery, robotics, surgical and medical robotics, skill learning, skill analysis and learning to perceive.

Results

Studies making use of ML methods in the context of surgery are increasingly being reported. In particular, there is an increasing interest in using ML for developing tools to understand and model surgical skill and competence or to extract surgical workflow. Many researchers begin to integrate this understanding into the control of recent surgical robots and devices.

Conclusion

ML is an expanding field. It is popular as it allows efficient processing of vast amounts of data for interpreting and real-time decision making. Already widely used in imaging and diagnosis, it is believed that ML will also play an important role in surgery and interventional treatments. In particular, ML could become a game changer into the conception of cognitive surgical robots. Such robots endowed with cognitive skills would assist the surgical team also on a cognitive level, such as possibly lowering the mental load of the team. For example, ML could help extracting surgical skill, learned through demonstration by human experts, and could transfer this to robotic skills. Such intelligent surgical assistance would significantly surpass the state of the art in surgical robotics. Current devices possess no intelligence whatsoever and are merely advanced and expensive instruments.

Keywords

Surgical robotics Skill learning Skill analysis  Learning to perceive 

Notes

Funding

This research has been funded by the European Commission’s 7th Framework Programme FP7-ICT, by the project CASCADE under Grant Agreement No.601021.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

References

  1. 1.
    Troccaz J (2008) Medical robotics: where we come from, where we are and where we could go. Ind Robot Int J 35(4)Google Scholar
  2. 2.
    Herron D, Marohn M (2008) A consensus document on robotic surgery. Surg Endosc 22(2):313–325CrossRefPubMedGoogle Scholar
  3. 3.
    Camberlin C, Senn A, Leys M, De Laet C (2009) Robot-assisted surgery: health technology assessment. KCE reports 104A. Federaal Kenniscentrum voor de GezondheidszorgGoogle Scholar
  4. 4.
    ElSahwi KS, Hooper C, Leon MCD, Gallo TN, Ratner E, Silasi D-A, Santin AD, Schwartz PE, Rutherford TJ, Azodi M (2012) Comparison between 155 cases of robotic vs. 150 cases of open surgical staging for endometrial cancer. Gynecol Oncol 124(2):260–264CrossRefPubMedGoogle Scholar
  5. 5.
    Brandao LF, Autorino R, Laydner H, Haber G-P, Ouzaid I, Sio MD, Perdonà S, Stein RJ, Porpiglia F, Kaouk JH (2014) Robotic versus laparoscopic adrenalectomy: a systematic review and meta-analysis. Eur Urol 65(6):1154–1161CrossRefPubMedGoogle Scholar
  6. 6.
    Morino M, Pellegrino L, Giaccone C, Garrone C, Rebecchi F (2006) Randomized clinical trial of robot-assisted versus laparoscopic nissen fundoplication. Br J Surg 93(5):553–558CrossRefPubMedGoogle Scholar
  7. 7.
    Nakadi I, Mélot C, Closset J, Moor V, Bétroune K, Feron P, Lingier P, Gelin M (2006) Evaluation of da Vinci nissen fundoplication clinical results and cost minimization. World J Surg 30(6):1050–1054CrossRefPubMedGoogle Scholar
  8. 8.
    Park JS, Choi G-S, Park SY, Kim HJ, Ryuk JP (2012) Randomized clinical trial of robot-assisted versus standard laparoscopic right colectomy. Br J Surg 99(9):1219–1226CrossRefPubMedGoogle Scholar
  9. 9.
    van den Berg J, Miller S, Duckworth D, Hu H, Wan A, Fu X, Goldberg K, Abbeel P (2010) Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 2074–2081Google Scholar
  10. 10.
    Kwoh Y, Hou J, Jonckheere E, Hayati S (1988) A robot with improved absolute positioning accuracy for CT guided stereotactic brain surgery. IEEE Trans Biomed Eng 35(2):153–160CrossRefPubMedGoogle Scholar
  11. 11.
    Paul H, Bargar W, Mittlestadt B, Musits B, Taylor R, Kazanzides P, Zuhars J, Williamson B, Hanson W (1992) Development of a surgical robot for cementless total hip arthroplasty. Clin Orthop Relat Res 285:57–66PubMedGoogle Scholar
  12. 12.
    Burckhardt CW, Flury P, Glauser D (1995) Stereotactic brain surgery. Eng Med Biol Mag IEEE 14(3):314–317CrossRefGoogle Scholar
  13. 13.
    Adler J, Chang S, Murphy M, Doty J, Geis P, Hancock S (1997) The Cyberknife: a frameless robotic system for radiosurgery. Stereotact Funct Neurosurg 69(1–4):124–128CrossRefPubMedGoogle Scholar
  14. 14.
    Kang H (2002) Robotic assisted suturing in minimally invasive surgery. Ph.D. thesis, Rensselaer Polytechnic Institute , Troy, New YorkGoogle Scholar
  15. 15.
    Jackson R, Cavusoglu M (2013) Needle path planning for autonomous robotic surgical suturing. In: IEEE international conference on robotics and automation (ICRA), pp 1669–1675Google Scholar
  16. 16.
    Nageotte F, Zanne P, Doignon C, de Mathelin M (2009) Stitching planning in laparoscopic surgery: towards robot-assisted suturing. Int J Robot Res 28(10):1303–1321CrossRefGoogle Scholar
  17. 17.
    Jansen R, Hauser K, Chentanez N, van der Stappen F, Goldberg K (2009) Surgical retraction of non-uniform deformable layers of tissue: 2D robot grasping and path planning. In: IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 4092–4097Google Scholar
  18. 18.
    Patil S, Alterovitz R (May 2010) Toward automated tissue retraction in robot-assisted surgery. In: IEEE international conference on robotics and automation (ICRA), pp 2088–2094Google Scholar
  19. 19.
    Muradore R, Bresolin D, Geretti L, Fiorini P, Villa T (2011) Robotic surgery—formal verification of plans. Robot Autom Mag IEEE 18(3):24–32CrossRefGoogle Scholar
  20. 20.
    Brett, P, Taylor R, Proops D, Coulson C, Reid A, Griffiths M (2007) A surgical robot for cochleostomy. In: Proceedings of 29th annual international conference of the IEEE engineering in medicine and biology society (EMBS), pp 1229–1232Google Scholar
  21. 21.
    Larsson J, Hayes-Roth B (1998) Guardian: intelligent autonomous agent for medical monitoring and diagnosis. Intell Syst Their Appl IEEE 13(1):58–64CrossRefGoogle Scholar
  22. 22.
    Frei C, Gentilini A, Derighetti M, Glattfelder A, Morari M, Schnider T, Zbinden A (1999) Automation in anesthesia. In: Proceedings of the American control conference (ACC), pp 1258–1263Google Scholar
  23. 23.
    Mitchell TM (1997) Machine learning. McGraw-Hill, New YorkGoogle Scholar
  24. 24.
    Bishop CM (2006) Pattern recognition and machine learning. Springer, BerlinGoogle Scholar
  25. 25.
    Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292Google Scholar
  26. 26.
    Price B, Boutilier C (2003) A Bayesian approach to imitation in reinforcement learning. In: Proceedings of the 18th international joint conference on artificial intelligence (IJCAI), pp 712–720Google Scholar
  27. 27.
    Birkhimer CE (2005) Extracting human strategies for use in robotic assembly. Ph.D. thesis, Case Western Reserve University, Cleveland, OH, USA. AAI3159443Google Scholar
  28. 28.
    Hannaford B, Lee P (1991) Hidden Markov model analysis of force/torque information in telemanipulation. Int J Robot Res 10(5):528–539CrossRefGoogle Scholar
  29. 29.
    Itabashi K, Hayakawa K, Suzuki T, Okuma S, Fujiwara F (1997) Modeling of the peg-in-hole task based on impedance parameters and HMM. Proc IEEE/RSJ Int Conf Intell Robots Syst 1:451–457Google Scholar
  30. 30.
    Asada H, Liu S (1991) Transfer of human skills to neural net robot controllers. Proc IEEE Int Conf Robot Autom 3:2442–2448Google Scholar
  31. 31.
    Kaiser M, Dillmann R (1996) Building elementary robot skills from human demonstration. Proc IEEE Int Conf Robot Autom 3:2700–2705CrossRefGoogle Scholar
  32. 32.
    Myers D (1999) An approach to automated programming of industrial robots. Proc IEEE Int Conf Robot Autom 4:3109–3114CrossRefGoogle Scholar
  33. 33.
    Skubic M, Volz R (2000) Acquiring robust, force-based assembly skills from human demonstration. IEEE Trans Robot Autom 16(6):772–781CrossRefGoogle Scholar
  34. 34.
    Ng AY, Russell SJ (2000) Algorithms for inverse reinforcement learning. In: Proceedings of the international conference on machine learning (ICML), pp 663–670Google Scholar
  35. 35.
    Abbeel P, Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the international conference on machine learning (ICML)Google Scholar
  36. 36.
    Ramachandran D, Amir E (2007) Bayesian inverse reinforcement learning. In: Proceedings of the international joint conference on artificial intelligence (IJCAI), pp 2586–2591Google Scholar
  37. 37.
    Kirk DE (1970) Optimal control theory: an introduction. Prentice-Hall, Englewood CliffsGoogle Scholar
  38. 38.
    Reznick RK, MacRae H (2006) Teaching surgical skills—changes in the wind. N Eng J Med 355(25):2664–2669CrossRefGoogle Scholar
  39. 39.
    Jog A, Itkowitz B, Liu M, DiMaio S, Hager G, Curet M, Kumar R (2011) Towards integrating task information in skills assessment for dexterous tasks in surgery and simulation. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 5273–5278. IEEEGoogle Scholar
  40. 40.
    Kenney PA, Wszolek MF, Gould JJ, Libertino JA, Moinzadeh A (2009) Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology 73(6):1288–1292CrossRefPubMedGoogle Scholar
  41. 41.
    Tedesco MM, Pak JJ, Harris EJ Jr, Krummel TM, Dalman RL, Lee JT (2008) Simulation-based endovascular skills assessment: the future of credentialing? J Vasc Surg 47(5):1008–1014CrossRefPubMedGoogle Scholar
  42. 42.
    Reiley CE, Lin HC, Yuh DD, Hager GD (2011) Review of methods for objective surgical skill evaluation. Surg Endosc 25(2):356–366CrossRefPubMedGoogle Scholar
  43. 43.
    Van Hove PD, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987CrossRefPubMedGoogle Scholar
  44. 44.
    Niitsu H, Hirabayashi N, Yoshimitsu M, Mimura T, Taomoto J, Sugiyama Y, Murakami S, Saeki S, Mukaida H, Takiyama W (2013) Using the objective structured assessment of technical skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg Today 43(3):271–275CrossRefPubMedPubMedCentralGoogle Scholar
  45. 45.
    Bridgewater B, Grayson AD, Jackson M, Brooks N, Grotte GJ, Keenan DJM, Millner R, Fabri BM, Mark J (2003) Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. Br Med J 327(7405):13–17CrossRefGoogle Scholar
  46. 46.
    Dosis A, Aggarwal R, Bello F, Moorthy K, Munz Y, Gillies D, Darzi A (2005) Synchronized video and motion analysis for the assessment of procedures in the operating theater. Arch Surg 140(3):293–299CrossRefPubMedGoogle Scholar
  47. 47.
    Schijven MP, Jakimowicz J, Schot C (2002) The advanced dundee endoscopic psychomotor tester (ADEPT) objectifying subjective psychomotor test performance. Surg Endosc 16(6):943–948CrossRefPubMedGoogle Scholar
  48. 48.
    Botden SMBI, Buzink SN, Schijven MP, Jakimowicz JJ (2008) ProMIS augmented reality training of laparoscopic procedures face validity. J Soc Simul Healthc 3(2):97–102CrossRefGoogle Scholar
  49. 49.
    Hattori M, Egi H, Tokunaga M, Suzuki T, Ohdan H, Kawahara T (July 2012) The integrated deviation in the HUESAD (Hiroshima University Endoscopic Surgical Assessment Device) represents the surgeon’s visual-spatial ability. In: Proceedings of The 2012 ICME international conference on complex medical engineering (CME), pp 316–320Google Scholar
  50. 50.
    van Empel PJ, van Rijssen LB, Commandeur JP, Verdam MGE, Huirne JA, Scheele F, Bonjer HJ, Meijerink WJ (2012) Validation of a new box trainer-related tracking device: the TrEndo. Surg Endosc 26:2346–2352CrossRefPubMedPubMedCentralGoogle Scholar
  51. 51.
    DiMaio SP, Hasser CJ (2008) The da Vinci research interface. In: MICCAI workshop on systems and architecture for computer assisted interventions, MIDAS JournalGoogle Scholar
  52. 52.
    Kazanzides P, DiMaio S, Deguet A, Vagvolgyi B, Balicki M, Schneider C, Kumar R, Jog A, Itkowitz B, Hasser C, Taylor R (2010) The surgical assistant workstation (SAW) in minimally-invasive surgery and microsurgery. In: Proceedings of MICCAI workshop on systems and architecture for computer assisted interventions (SACAI)Google Scholar
  53. 53.
    Rosen J, Brown JD, Chang L, Sinanan MN, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Trans Biomed Eng 53(3):399–413CrossRefPubMedGoogle Scholar
  54. 54.
    Speidel S, Zentek T, Sudra G, Gehrig T, Müller-Stich BP, Gutt C, Dillmann R (2009) Recognition of surgical skills using hidden markov models. In: Proceedings of SPIE medical imaging, vol 7261, p 25Google Scholar
  55. 55.
    Lin HC, Shafran I, Murphy TE, Okamura AM, Yuh DD, Hager GD (2005) Automatic detection and segmentation of robot-assisted surgical motions. In: Proceedings of medical image computing and computer-assisted intervention (MICCAI), Springer, pp 802–810Google Scholar
  56. 56.
    Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB (2011) Observational tools for assessment of procedural skills: a systematic review. Am J Surg 202(4):469–480CrossRefPubMedGoogle Scholar
  57. 57.
    Shah CV, Riga CV, Stoyanov D, Yang G-Z, Cheshire NJW, Bicknell CD (2011) Video motion analysis for approaching objective assessment of catheter-based endovascular interventions. In: Proceedings of The Hamlyn symposium on medical robotics, pp 93–94Google Scholar
  58. 58.
    Neequaye SK, Aggarwal R, Van Herzeele I, Darzi A, Cheshire NJ (2007) Endovascular skills training and assessment. J Vasc Surg 46(5):1055–1064CrossRefPubMedGoogle Scholar
  59. 59.
    Reiley CE, Hager GD (2009) Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In: Proceedings of medical image computing and computer-assisted intervention (MICCAI), Springer, pp 435–442Google Scholar
  60. 60.
    Lin H, Shafran I, Yuh D, Hager G (2006) Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg 11(5):220–230CrossRefPubMedGoogle Scholar
  61. 61.
    Reiley C, Lin H, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh D, Hager (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inf 132:396–401Google Scholar
  62. 62.
    Rafii-Tari H, Liu J, Lee S-L, Bicknell C, Yang G-Z (2013) Learning-based modeling of endovascular navigation for collaborative robotic catheterization. In: Proceedings of medical image computing and computer-assisted intervention (MICCAI), vol 8150, Springer, pp 369–377Google Scholar
  63. 63.
    Padoy N, Blum T, Essa I, Feussner H, Berger M-O, Navab N (2007) A boosted segmentation method for surgical workflow analysis. In: Medical image computing and computer-assisted intervention (MICCAI), vol 4791. Lecture Notes in Computer Science, Springer, Berlin Heidelberg, pp 102–109Google Scholar
  64. 64.
    Ahmadi S-A, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2006) Recovery of surgical workflow without explicit models. In: Medical image computing and computer-assisted intervention (MICCAI), vol 4190. Springer, pp 420–428Google Scholar
  65. 65.
    Reiley C, Hager G (2009) Task versus subtask surgical skill evaluation of robotic minimally invasive surgery. In: Medical image computing and computer-assisted intervention (MICCAI), vol 5761. Springer, pp 435–442Google Scholar
  66. 66.
    Varadarajan B, Reiley C, Lin H, Khudanpur S, Hager G (2009) Data-derived models for segmentation with application to surgical assessment and training. In: Medical image computing and computer-assisted intervention (MICCAI), vol 5761, Springer, pp 426–434Google Scholar
  67. 67.
    Varadarajan B (2011) Learning and inference algorithms for dynamical system models of dextrous motion. Ph.D. thesis, The Johns Hopkins UniversityGoogle Scholar
  68. 68.
    Tao L, Elhamifar E, Khudanpur S, Hager GD, Vidal R (2012) Sparse hidden markov models for surgical gesture classification and skill evaluation. In: Proceedings of information processing in computer-assisted interventions (IPCAI), Springer, pp 167–177Google Scholar
  69. 69.
    Rosen J, Brown J, Chang L, Sinanan M, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete markov model. IEEE Trans Biomed Eng 53(3):399–413CrossRefPubMedGoogle Scholar
  70. 70.
    Moustris GP, Hiridis SC, Deliparaschos KM, Konstantinidis KM (2011) Evolution of autonomous and semi-autonomous robotic surgical systems: a review of the literature. Int J Med Robot Comput Assist Surg 7(4):375–392CrossRefGoogle Scholar
  71. 71.
    Drake J, Joy M, Goldenberg A, Kreindler D (1991) Computer and robotic assisted resection of brain tumours. In: Proceedings of the fifth international conference on advanced robotics (ICAR), ’Robots in Unstructured Environments’, vol 1, pp 888–892Google Scholar
  72. 72.
    De Momi E, Ferrigno G (2010) Robotic and artificial intelligence for keyhole neurosurgery: the robocast project, a multi-modal autonomous path planner. Proc Inst Mech Eng H 224(5):715–727CrossRefPubMedGoogle Scholar
  73. 73.
    Benabid A, Cinquin P, Lavalle S, Bas JL, Demongeot J, de Rougemont J (1987) Computer-driven robot for stereotactic surgery connected to CT scan and magnetic resonance imaging. Technological design and preliminary results. Stereotact Funct Neurosurg 50(1–6):153–154CrossRefGoogle Scholar
  74. 74.
    Taylor R, Paul H, Mittelstadt B, Hanson W, Kazanzides P, Zuhars J, Glassman E, Musits B, Williamson W, Bargar W (1990) An image-directed robotic system for precise orthopaedic surgery. In: Proceedings of the 12th annual international conference of the IEEE on engineering in medicine and biology society (EMBC), pp 1928–1930Google Scholar
  75. 75.
    Taylor RH, Mittelstadt BD, Paul HA, Hanson W, Kazanzides P, Zuhars JF, Williamson B, Musits BL, Glassman E, Bargar WL (1994) An image-directed robotics system for precise orthopedic surgery. IEEE Trans Robot Autom 10(3):261–275CrossRefGoogle Scholar
  76. 76.
    Kazanzides P, Mittelstadt B, Musits B, Bargar W, Zuhars J, Williamson B, Cain P, Carbone E (1995) An integrated system for cementless hip replacement. Eng Med Biol Mag IEEE 14(3):307–313CrossRefGoogle Scholar
  77. 77.
    Fu L, Du Z, Sun L (2004) A novel robot-assisted bonesetting system. Proc IEEE/RSJ Int Conf Intell Robots Syst 3:2247–2252Google Scholar
  78. 78.
    Mönnich H, Stein D, Raczkowsky J, Worn H (May 2010) An automatic and complete self-calibration method for robotic guided laser ablation. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 1086–1087Google Scholar
  79. 79.
    Schorr O, Münchenberg J, Raczkowsky J, Wörn H (2001) KasOp—a generic system for pre- and intraoperative surgical assistance and guidance. In: Proceedings of the 15th international congress and exhibition of computer assisted radiology and surgery (CARS)Google Scholar
  80. 80.
    Engel D, Raczkowsky J, Wörn H (2001) A safe robot system for craniofacial surgery. In: Proceedings IEEE international conference on robotics and automation (ICRA), vol 2, pp 2020–2024Google Scholar
  81. 81.
    Coulson CJ, Reid AP, Proops DW (2008) A cochlear implantation robot in surgical practice. In: Proceedings of the 15th international conference on mechatronics and machine vision in practice (M2VIP), pp 173–176Google Scholar
  82. 82.
    Weber S, Belle B, Brett P, Du X, Caversaccio M, Proops D, Coulson C, Reid A (2013) Minimally invasive, robot assisted cochlear implantation. In: Proceedings of the 3rd joint workshop on new technologies for computer/robot assisted surgery (CRAS), pp 134–137Google Scholar
  83. 83.
    Coulson C, Taylor R, Reid A, Griffiths M, Proops D, Brett P (2008) An autonomous surgical robot for drilling a cochleostomy: preliminary porcine trial. Clin Otolaryngol 33(4):343–347CrossRefPubMedGoogle Scholar
  84. 84.
    Adler J, Murphy M, Martin J, Chang S, Hancock S (1999) Image-guided robotic radiosurgery. Neurosurgery 44(6):1299–1306PubMedGoogle Scholar
  85. 85.
    Ernst F, Schweikard A (2009) Forecasting respiratory motion with accurate online support vector regression (SVRpred). Int J Comput Assist Radiol Surg 4(5):439–447CrossRefPubMedGoogle Scholar
  86. 86.
    Kassahun Y, Yu B, Vander Poorten E (2013) Learning catheter-aorta interaction model using joint probability densities. In: Proceedings of the 3rd joint workshop on new technologies for computer/robot assisted surgery, pp 158–160Google Scholar
  87. 87.
    Bauernschmitt R, Schirmbeck EU, Knoll A, Mayer H, Nagy I, Wessel N, Wildhirt SM, Lange R (2005) Towards robotic heart surgery: introduction of autonomous procedures into an experimental surgical telemanipulator system. Int J Med Robot Comput Assist Surg 1(3):74–79CrossRefGoogle Scholar
  88. 88.
    Ji C, Hou Z-G, Xie X-L (2011) An image-based guidewire navigation method for robot-assisted intravascular interventions. In: Engineering in medicine and biology society, EMBC, 2011 annual international conference of the IEEE, pp 6680–6685. IEEEGoogle Scholar
  89. 89.
    Tercero C, Ikeda S, Uchiyama T, Fukuda T, Arai F, Ono Y (2006) Catheter insertion mechanism and feedback control using magnetic motion capture sensor. In: Proceedings of international joint conference on SICE-ICASE, pp 1856–1859Google Scholar
  90. 90.
    Tercero Villagran C, Ikeda S, Fukuda T, Sekiyama K, Okada Y, Uchiyama T, Negoro M, Takahashi I (2007) Catheter insertion path reconstruction with autonomous system for endovascular surgery. In: Proceedings of international symposium on computational intelligence in robotics and automation (CIRA), pp 398–403. IEEEGoogle Scholar
  91. 91.
    Tercero C, Ikeda S, Uchiyama T, Fukuda T, Arai F, Okada Y, Ono Y, Hattori R, Yamamoto T, Negoro M, Takahashi I (2007) Autonomous catheter insertion system using magnetic motion capture sensor for endovascular surgery. Int J Med Robot Comput Assist Surg 3:52–58CrossRefGoogle Scholar
  92. 92.
    Harris S, Arambula-Cosio F, Mei Q, Hibberd R, Davies B, Wickham J, Nathan M, Kundu B (1997) The probot—an active robot for prostate resection. Proc Inst Mech Eng Part H J Eng Med 211(4):317–325CrossRefGoogle Scholar
  93. 93.
    Muntener M, Patriciu A, Petrisor D, Mazilu D, Bagga H, Kavoussi L, Cleary K, Stoianovici D (2006) Magnetic resonance imaging compatible robotic system for fully automated brachytherapy seed placement. Urology 68(6):1313–1317CrossRefPubMedPubMedCentralGoogle Scholar
  94. 94.
    Patriciu A, Petrisor D, Muntener M, Mazilu D, Schar M, Stoianovici D (2007) Automatic brachytherapy seed placement under MRI guidance. IEEE Trans Biomed Eng 54(8):1499–1506CrossRefPubMedPubMedCentralGoogle Scholar
  95. 95.
    Davies B, Hibberd R, Coptcoat M (1989) A surgeon robot for prostatectomy—a laboratory evaluation. J Med Eng Technol 13:273–277CrossRefPubMedGoogle Scholar
  96. 96.
    Casals A, Amat J, Laporte E (1996) Automatic guidance of an assistant robot in laparoscopic surgery. Proc IEEE Int Conf Robot Autom 1:895–900CrossRefGoogle Scholar
  97. 97.
    Wei G-Q, Arbter K, Hirzinger G (1997) Real-time visual servoing for laparoscopic surgery. Controlling robot motion with color image segmentation. Eng Med Biol Mag IEEE 16(1):40–45CrossRefGoogle Scholar
  98. 98.
    Krupa A, Doignon C, Gangloff J, Mathelin M, Solert L, Morel G (2001) Towards semi-autonomy in laparoscopic surgery through vision and force feedback control. In: Rus D, Singh S (eds) Experimental Robotics VII, vol 271, Lecture Notes in Control and Information Sciences. Springer, Berlin Heidelberg, pp 189–198Google Scholar
  99. 99.
    Krupa A, Gangloff J, Doignon C, de Mathelin M, Morel G, Leroy J, Soler L, Marescaux J (2003) Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE Trans Robot Autom 19(5):842–853CrossRefGoogle Scholar
  100. 100.
    Weede O, Mönnich H, Müller B, Wörn H (2011) An intelligent and autonomous endoscopic guidance system for minimally invasive surgery. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 5762–5768Google Scholar
  101. 101.
    Krupa A, Gangloff J, de Mathelin M, Doignon C, Morel G, Soler L, Leroy J, Marescaux J (2002) Autonomous retrieval and positioning of surgical instruments in robotized laparoscopic surgery using visual servoing and laser pointers. In: Proceedings of IEEE international conference on robotics and automation (ICRA), vol 4, pp 3769–3774Google Scholar
  102. 102.
    Taguchi K, Ohnishi K (2008) A bilateral teleoperation method using an autonomous control based on DFT modal decomposition. In: Proceedings of the 34th annual conference of IEEE on industrial electronics (IECON), pp 2497–2502. IEEEGoogle Scholar
  103. 103.
    Estebanez B, Bauzano E, Muoz V (2011) Surgical tools pose estimation for a multimodal hmi of a surgical robotic assistant. In: 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 2121–2126Google Scholar
  104. 104.
    Ko S-Y, Kwon D-S (2004) A surgical knowledge based interaction method for a laparoscopic assistant robot. In: Proceedings of 13th IEEE international workshop on robot and human interactive communication (ROMAN), pp 313–318. IEEEGoogle Scholar
  105. 105.
    Gangloff J, Ginhoux R, De Mathelin M, Soler L, Marescaux J (2006) Model predictive control for compensation of cyclic organ motions in teleoperated laparoscopic surgery. IEEE Trans Control Syst Technol 14(2):235–246CrossRefGoogle Scholar
  106. 106.
    Bauzano E, Mu andoz V, Garcia-Morales I (2010) Auto-guided movements on minimally invasive surgery for surgeon assistance. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 1843–1848. IEEEGoogle Scholar
  107. 107.
    Osa T, Staub C, Knoll A (2010) Framework of automatic robot surgery system using visual servoing. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 1837–1842Google Scholar
  108. 108.
    Staub C, Knoll A, Osa T, Bauernschmitt R (March 2010) Autonomous high precision positioning of surgical instruments in robot-assisted minimally invasive surgery under visual guidance. In: Proceedings of sixth international conference on autonomic and autonomous systems (ICAS), pp 64–69Google Scholar
  109. 109.
    Dumpert J, Lehman A, Wood N, Oleynikov D, Farritor S (2009) Semi-autonomous surgical tasks using a miniature in vivo surgical robot. In: Annual international conference of IEEE on engineering in medicine and biology society (EMBC), pp 266–269. IEEEGoogle Scholar
  110. 110.
    King B, Reisner L, Pandya A, Composto A, Ellis R, Klein M (2013) Towards an autonomous robot for camera control during laparoscopy surgery. J Laparoendosc Adv Surg Tech 23(12):1027–1030CrossRefGoogle Scholar
  111. 111.
    Bauzano E, Garcia-Morales I, del Saz-Orozco P, Fraile J, Muoz V (2013) A minimally invasive surgery robotic assistant for HALS–SILS techniques. Comput Methods Prog Biomed 112(2):272–283CrossRefGoogle Scholar
  112. 112.
    Cornella J, Elle O-J, Ali W, Samset E (2008) Intraoperative navigation of an optically tracked surgical robot. Med Image Comput Assist Interv 11(2):587–594Google Scholar
  113. 113.
    Chen G, Pham MT, Redarce T (2008) A semi-autonomous micro-robotic system for colonoscopy. In: Proceedings of IEEE international conference on robotics and biomimetics (ROBIO), pp 703–708. IEEEGoogle Scholar
  114. 114.
    Kang H, Wen J (2000) Autonomous suturing using minimally invasive surgical robots. In: Proceedings of IEEE international conference on control applications (CCA), pp 742–747Google Scholar
  115. 115.
    Mayer H, Gomez F, Wierstra D, Nagy I, Knoll A, Schmidhuber J (2006) A system for robotic heart surgery that learns to Tie Knots using recurrent neural networks. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 543–548Google Scholar
  116. 116.
    Hynes P, Dodds G, Wilkinson AJ (2006) Uncalibrated visual-servoing of a dual-arm robot for MIS Suturing. In: Proceedings of the first IEEE/RAS-EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 420–425Google Scholar
  117. 117.
    Mayer HG, Nagy I, Knoll A, Schirmbeck EU, Bauernschmitt R (2004) The EndoPAR system for minimally invasive robotic surgery. Proc IEEE/RSJ Int Conf Intell Robots Syst 4:3637–3642Google Scholar
  118. 118.
    Mayer H, Wierstra D, Gomez F, Nagy I, Knoll A, Schmidhuber J (2008) A system for robotic heart surgery that learns to tie knots using recurrent neural networks. Adv Robot 22(13–14):1521–1537CrossRefGoogle Scholar
  119. 119.
    Mayer H, Nagy I, Burschka D, Knoll A, Braum E, Lange R, Bauernschmitt R (2008) Automation of manual tasks for minimally invasive surgery. In: Proceedings of international conference on autonomic and autonomous systems (ICAS), pp 260–265Google Scholar
  120. 120.
    Chatelain P, Krupa A, Marchal M (2013) Real-time needle detection and tracking using a visually servoed 3D ultrasound probe. In: IEEE international conference on robotics and automation (ICRA), pp 1676–1681. IEEEGoogle Scholar
  121. 121.
    Mayer H, Nagy I, Knoll A, Braun E, Lange R, Bauernschmitt R (2007) Adaptive control for human-robot skilltransfer: trajectory planning based on fluid dynamics. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 1800–1807. IEEEGoogle Scholar
  122. 122.
    Reiley C, Plaku E, Hager G (2010) Motion generation of robotic surgical tasks: Learning from expert demonstrations. In: Proceedings of annual international conference of the IEEE on engineering in medicine and biology society (EMBC), pp 967–970. IEEEGoogle Scholar
  123. 123.
    Chow D-L, Newman W (2013) Improved knot-tying methods for autonomous robot surgery. In: Proceedings of IEEE international conference on automation science and engineering (CASE), pp 461–465. IEEEGoogle Scholar
  124. 124.
    Kang H, Wen J (2001) Endobot: a robotic assistant in minimally invasive surgeries. In: IEEE international conference on robotics and automation, 2001. Proceedings 2001 ICRA, vol 2, pp 2031–2036. IEEEGoogle Scholar
  125. 125.
    Schulman J, Ho J, Lee C, Abbeel P (2013) Learning from Demonstrations through the use of non-rigid registration. In: Proceedings of the 16th international symposium on robotics research (ISRR)Google Scholar
  126. 126.
    Kang H, Wen J (2001) Robotic assistants aid surgeons during minimally invasive procedures. Eng Med Biol Mag IEEE 20(1):94–104CrossRefGoogle Scholar
  127. 127.
    Kantor GS (2015) Intravenous catheter complications. http://www.netwellness.org/healthtopics/anesthesiology/ivcomplications.cfm. [Online; Accessed 16 Sep 2015]
  128. 128.
    Zong G, Hu Y, Li D, Sun X (2006) Visually servoed suturing for robotic microsurgical keratoplasty. In: 2006 IEEE/RSJ international conference on intelligent robots and systems, pp 2358–2363. IEEEGoogle Scholar
  129. 129.
    Mallapragada V, Sarkar N, Podder T (2008) Autonomous coordination of imaging and tumor manipulation for robot assisted breast biopsy. In: Proceedings of 2nd IEEE RAS EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 676–681. IEEEGoogle Scholar
  130. 130.
    Mallapragada V, Sarkar N, Podder T (2011) Toward a robot-assisted breast intervention system. IEEE/ASME Trans Mechatron 16(6):1011–1020CrossRefGoogle Scholar
  131. 131.
    Liang K, Light E, Rogers A, Von Allmen D, Smith S (2009) 3-D ultrasound guidance of autonomous surgical robotics: feasibility studies. In: Proceedings of IEEE international ultrasonics symposium (IUS), pp 582–585. IEEEGoogle Scholar
  132. 132.
    Liang K, Rogers A, Light E, von Allmen D, Smith S (2010) Three-dimensional ultrasound guidance of autonomous robotic breast biopsy: feasibility study. Ultrasound Med Biol 36(1):173–177CrossRefPubMedPubMedCentralGoogle Scholar
  133. 133.
    Bonfe M, Preda N, Secchi C, Ferraguti F, Muradore R, Repele L, Lorenzi G, Gasperotti L, Fiorini P (2013) Automated surgical task execution: the needle insertion case. In: Proceedings of 3rd joint workshop on new technologies for computer/robot assisted surgery (CRAS), pp 45–47Google Scholar
  134. 134.
    Whitman J, Fronheiser MP, Ivancevich NM, Smith SW (2007) Autonomous surgical robotics using 3-d ultrasound guidance: feasibility study. Ultrason Imag 29(4):213–219CrossRefGoogle Scholar
  135. 135.
    Staub C, Osa T, Knoll A, Bauernschmitt R (2010) Automation of tissue piercing using circular needles and vision guidance for computer aided laparoscopic surgery. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 4585–4590Google Scholar
  136. 136.
    Zivanovic A, Davies BL (2000) A robotic system for blood sampling. Inf Technol Biomed IEEE Trans 4(1):8–14CrossRefGoogle Scholar
  137. 137.
    Osa T, Haniu T, Harada K, Sugita N, Mitsuishi M (2013) Perforation risk detector using demonstration-based learning for teleoperated robotic surgery. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 2572–2577. IEEEGoogle Scholar
  138. 138.
    Nichols K, Okamura A (2013) Autonomous robotic palpation: Machine learning techniques to identify hard inclusions in soft tissues. In: 2013 IEEE International Conference on robotics and automation (ICRA), pp 4384–4389. IEEEGoogle Scholar
  139. 139.
    Abolmaesumi P, Salcudean S, Zhu W-H, Sirouspour M, DiMaio S (2002) Image-guided control of a robot for medical ultrasound. IEEE Trans Robot Autom 18(1):11–23CrossRefGoogle Scholar
  140. 140.
    Li T, Kermorgant O, Krupa A (2012) Maintaining visibility constraints during tele-echography with ultrasound visual servoing. In: IEEE international conference on robotics and automation (ICRA), pp 4856–4861. IEEEGoogle Scholar
  141. 141.
    Gumprecht JDJ, Baumann M, Menz A, Stolzenburg J-U, Lueth T (2012) A modular control concept for a flat-panel ultrasound robot. In: 2012 4th IEEE RAS EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 907–912. IEEEGoogle Scholar
  142. 142.
    Mylonas GP, Giataganas P, Chaudery M, Vitiello V, Darzi A, Yang G-Z (2013) Autonomous eFAST ultrasound scanning by a robotic manipulator using learning from demonstrations. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 3251–3256. IEEEGoogle Scholar
  143. 143.
    Giataganas P, Vitiello V, Simaiaki V, Lopez E, Yang G-Z (2013) Cooperative in situ microscopic scanning and simultaneous tissue surface reconstruction using a compliant robotic manipulator. In: Proceedings of IEEE international conference on robotics and automation (ICRA), pp 5378–5383. IEEEGoogle Scholar
  144. 144.
    Fichera L, Pardo D, Mattos LS (2013) Modeling tissue temperature dynamics during laser exposure. In: Proceedings of the international work conference on artificial neural networks (IWANN), pp 96–106Google Scholar
  145. 145.
    Kehoe B, Kahn G, Mahler J, Kim J, Lee A, Lee A, Nakagawa K, Patil S, Boyd WD, Abbeel P, Goldberg K (2014) Autonomous multilateral debridement with the raven surgical robot. In: Proceedings of the IEEE international conference on robotics and automation (ICRA)Google Scholar
  146. 146.
    Garcia P, Rosen J, Kapoor C, Noakes M, Elbert G, Treat M, Ganous T, Hanson M, Manak J, Hasser C, Rohler D, Satava R (2009) Trauma pod: a semi-automated telerobotic surgical system. Int J Med Robot Comput Assist Surg 5(2):136–146CrossRefGoogle Scholar
  147. 147.
    Treat M, Amory S, Downey P, Taliaferro D (2006) Initial clinical experience with a partly autonomous robotic surgical instrument server. Surg Endosc Other Interv Tech 20(8):1310–1314CrossRefGoogle Scholar
  148. 148.
    Mönnich H, Botturi D, Raczko Raczkowsky H, Wörn H (2009) System architecture for workflow controlled robotic surgery. J Inf Technol Healthc 6Google Scholar
  149. 149.
    Bonfe M, Boriero F, Dodi R, Fiorini P, Morandi A, Muradore R, Pasquale L, Sanna A, Secchi C (2012) Towards automated surgical robotics: a requirements engineering approach. In: Proceedings of the 4th IEEE RAS EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 56–61Google Scholar
  150. 150.
    Bresolin D, Di Guglielmo L, Geretti L, Muradore R, Fiorini P, Villa T (2012) Open problems in verification and refinement of autonomous robotic systems. In: Proceedings of the 15th euromicro conference on digital system design (DSD), pp 469–476. IEEEGoogle Scholar
  151. 151.
    Muradore R, Zerbato D, Vezzaro L, Gasperotti L, Fiorini P (2012) From simulation to abstract modeling of surgical operations. In: Proceedings of the 2nd joint workshop on new technologies for computer/robot assisted surgery (CRAS), MadridGoogle Scholar
  152. 152.
    Botturi D, Fiorini P (2005) Optimal control for autonomous task execution. In: 44th IEEE conference on decision and control, 2005 and 2005 European control conference. CDC-ECC ’05, pp 3525–3530. IEEEGoogle Scholar
  153. 153.
    Monnich H, Worn H, Stein D (2012) OP sense—a robotic research platform for telemanipulated and automatic computer assisted surgery. In: Proceedings of 12th IEEE international workshop on advanced motion control (AMC), pp 1–6. IEEEGoogle Scholar
  154. 154.
    Ginhoux R, Gangloff J, De Mathelin M, Soler L, Sanchez M, Marescaux J (2005) Active filtering of physiological motion in robotized surgery using predictive control. IEEE Trans Robot 21(1):67–79CrossRefGoogle Scholar
  155. 155.
    Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRefPubMedGoogle Scholar
  156. 156.
    Calinon S, Bruno D, Malekzadeh MS, Nanayakkara T, Caldwell DG (2014) Human-robot skills transfer interfaces for a flexible surgical robot. Comput Methods Prog Biomed 116(2):81–96CrossRefGoogle Scholar
  157. 157.
    Maron JK, Zenati MA (2013) Framework for managing cognitive load in a data-rich robotic operating room. In: Proceedings of the 3rd joint workshop on new technologies for computer/robot assisted surgery (CRAS), pp 150–153Google Scholar
  158. 158.
    Endsley MR (1995) Toward a theory of situation awareness in dynamic system. Hum Factors J Hum Factors Ergon Soc 37(1):32–64CrossRefGoogle Scholar
  159. 159.
    Jacob MG, Li Y-T, Akingba GA, Wachs JP (2013) Collaboration with a robotic scrub nurse. Commun ACM 56(5):68–75CrossRefGoogle Scholar
  160. 160.
    Meyfroidt G, Güizab F, Ramonb J, Bruynooghe M (2009) Machine learning techniques to examine large patient databases. Best Pract Res Clin Anaesthesiol 23(1):127–143CrossRefPubMedGoogle Scholar
  161. 161.
    Petersen J, Rodriguez Baena F (Nov 2013) A dynamic active constraints approach for hands-on robotic surgery. In: 2013 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 1966–1971Google Scholar

Copyright information

© CARS 2015

Authors and Affiliations

  • Yohannes Kassahun
    • 1
    Email author
  • Bingbin Yu
    • 2
  • Abraham Temesgen Tibebu
    • 2
  • Danail Stoyanov
    • 3
  • Stamatia Giannarou
    • 4
  • Jan Hendrik Metzen
    • 2
  • Emmanuel Vander Poorten
    • 5
  1. 1.Robotics Innovation CenterGerman Research Center for Artificial IntelligenceBremenGermany
  2. 2.Faculty 3 - Mathematics and Computer ScienceUniversity of BremenBremenGermany
  3. 3.Centre for Medical Image Computing, Department of Computer ScienceUniversity College LondonLondonUK
  4. 4.Hamlyn Centre for Robotic SurgeryImperial College LondonLondonUK
  5. 5.Department of Mechanical EngineeringUniversity of LeuvenHeverleeBelgium

Personalised recommendations