Skip to main content
SpringerLink
Log in

Review of surgical robotic systems for keyhole and endoscopic procedures: state of the art and perspectives

  • Review
  • Published:
Frontiers of Medicine Aims and scope Submit manuscript

Abstract

Minimally invasive surgery, including laparoscopic and thoracoscopic procedures, benefits patients in terms of improved postoperative outcomes and short recovery time. The challenges in hand-eye coordination and manipulation dexterity during the aforementioned procedures have inspired an enormous wave of developments on surgical robotic systems to assist keyhole and endoscopic procedures in the past decades. This paper presents a systematic review of the state-of-the-art systems, picturing a detailed landscape of the system configurations, actuation schemes, and control approaches of the existing surgical robotic systems for keyhole and endoscopic procedures. The development challenges and future perspectives are discussed in depth to point out the need for new enabling technologies and inspire future researches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Cuschieri A. Laparoscopic surgery: current status, issues and future developments. Surgeon 2005; 3(3): 125–130, 132–133, 135–138

    CAS  PubMed  Google Scholar 

  2. Alimoglu O, Sagiroglu J, Atak I, Kilic A, Eren T, Caliskan M, Bas G. Robot-assisted laparoscopic (RAL) procedures in general surgery. Int J Med Robot 2016; 12(3): 427–430

    PubMed  Google Scholar 

  3. Taylor RH. A perspective on medical robotics. Proc IEEE 2006; 94(9): 1652–1664

    Google Scholar 

  4. Kuo CH, Dai JS, Dasgupta P. Kinematic design considerations for minimally invasive surgical robots: an overview. Int J Med Robot 2012; 8(2): 127–145

    PubMed  Google Scholar 

  5. Bergeles C, Yang GZ. From passive tool holders to microsurgeons: safer, smaller, smarter surgical robots. IEEE Trans Biomed Eng 2014; 61(5): 1565–1576

    PubMed  Google Scholar 

  6. Smith JA, Jivraj J, Wong R, Yang V. 30 years of neurosurgical robots: review and trends for manipulators and associated navigational systems. Ann Biomed Eng 2016; 44(4): 836–846

    PubMed  Google Scholar 

  7. Jacofsky DJ, Allen M. Robotics in arthroplasty: a comprehensive review. J Arthroplasty 2016; 31(10): 2353–2363

    PubMed  Google Scholar 

  8. Kim C, Ryu SC, Dupont PE. Real-time adaptive kinematic model estimation of concentric tube robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2015: 3214–3219

  9. Bergeles C, Gosline AH, Vasilyev NV, Codd PJ, Del Nido PJ, Dupont PE. Concentric tube robot design and optimization based on task and anatomical constraints. IEEE Trans Robot 2015; 31(1): 67–84

    PubMed  PubMed Central  Google Scholar 

  10. Elhawary H, Tse ZTH, Hamed A, Rea M, Davies BL, Lamperth MU. The case for MR-compatible robotics: a review of the state of the art. Int J Med Robot 2008; 4(2): 105–113

    PubMed  Google Scholar 

  11. Burgner-Kahrs J, Rucker DC, Choset H. Continuum robots for medical applications: a survey. IEEE Trans Robot 2015; 31(6): 1261–1280

    Google Scholar 

  12. Beasley RA. Medical robots: current systems and research directions. J Robot 2012; 2012: 1–14

    Google Scholar 

  13. Diana M, Marescaux J. Robotic surgery. Br J Surg 2015; 102(2): e15–e28

    CAS  PubMed  Google Scholar 

  14. Rassweiler JJ, Autorino R, Klein J, Mottrie A, Goezen AS, Stolzenburg JU, Rha KH, Schurr M, Kaouk J, Patel V, Dasgupta P, Liatsikos E. Future of robotic surgery in urology. BJU Int 2017; 120(6): 822–841

    PubMed  Google Scholar 

  15. Navarra G, Pozza E, Occhionorelli S, Carcoforo P, Donini I. One-wound laparoscopic cholecystectomy. Br J Surg 1997; 84(5): 695

    CAS  PubMed  Google Scholar 

  16. Kalloo AN, Singh VK, Jagannath SB, Niiyama H, Hill SL, Vaughn CA, Magee CA, Kantsevoy SV. Flexible transgastric peritoneoscopy: a novel approach to diagnostic and therapeutic interventions in the peritoneal cavity. Gastrointest Endosc 2004; 60(1): 114–117

    PubMed  Google Scholar 

  17. Guthart GS, Salisbury JK. The IntuitiveTM Telesurgery System: overview and application. IEEE International Conference on Robotics and Automation (ICRA) 2000: 618–621

  18. Abdel Raheem A, Troya IS, Kim DK, Kim SH, Won PD, Joon PS, Hyun GS, Rha KH. Robot-assisted Fallopian tube transection and anastomosis using the new REVO-I robotic surgical system: feasibility in a chronic porcine model. BJU Int 2016; 118(4): 604–609

    Google Scholar 

  19. Taylor R, Jensen P, Whitcomb L, Barnes A, Kumar R, Stoianovici D, Gupta P, Wang Z, Dejuan E, Kavoussi L. A steady-hand robotic system for microsurgical augmentation. Int J Robot Res 1999; 18(12): 1201–1210

    Google Scholar 

  20. Li J, Zhou N, Wang S, Gao Y, Liu D. Design of an integrated master-slave robotic system for minimally invasive surgery. Int J Med Robot 2012; 8(1): 77–84

    CAS  PubMed  Google Scholar 

  21. Freschi C, Ferrari V, Melfi F, Ferrari M, Mosca F, Cuschieri A. Technical review of the da Vinci surgical telemanipulator. Int J Med Robot 2013; 9(4): 396–406

    CAS  PubMed  Google Scholar 

  22. Kim U, Lee DH, Kim YB, Seok DY, So J, Choi HR. S-Surge: novel portable surgical robot with multiaxis force-sensing capability for minimally invasive surgery. IEEE/ASME Trans Mechatron 2017; 22(4): 1717–1727

    Google Scholar 

  23. Lum MJH, Friedman DCW, Sankaranarayanan G, King H, Fodero K, Leuschke R, Hannaford B, Rosen J, Sinanan MN. The RAVEN: design and validation of a telesurgery system. Int J Robot Res 2009; 28(9): 1183–1197

    Google Scholar 

  24. Hannaford B, Rosen J, Friedman DW, King H, Roan P, Cheng L, Glozman D, Ma J, Kosari SN, White L. Raven-II: an open platform for surgical robotics research. IEEE Trans Biomed Eng 2013; 60(4): 954–959

    PubMed  Google Scholar 

  25. Kuo CH, Dai JS. Kinematics of a fully-decoupled remote center-of-motion parallel manipulator for minimally invasive surgery. J Med Device 2012; 6(2): 021008

    Google Scholar 

  26. Mitsuishi M, Sugita N, Pitakwatchara P. Force feedback augmentation modes in the laparoscopic minimal invasive telesurgical system. IEEE/ASME Trans Mechatron 2007; 12(4): 447–454

    Google Scholar 

  27. Berkelman P, Ma J. A compact modular teleoperated robotic system for laparoscopic surgery. Int J Robot Res 2009; 28(9): 1198–1215

    Google Scholar 

  28. Dombre E, Michelin M, Pierrot F, Poignet P, Bidaud P, Morel G, Ortmaier T, Salle D, Zemiti N, Gravez P, Karouia M, Bonnet N. MARGE project: design, modelling, and control of assistive devices for minimally invasive surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2004; LNCS 3217: 1–8

  29. Hagn U, Nickl M, Jörg S, Passig G, Bahls T, Nothhelfer A, Hacker F, Le-Tien L, Albu-Schäffer A, Konietschke R, Grebenstein M, Warpup R, Haslinger R, Frommberger M, Hirzinger G. The DLR MIRO: a versatile lightweight robot for surgical applications. Industrial Robot 2008; 35(4): 324–336

    Google Scholar 

  30. Lopez E, Kwok KW, Payne CJ, Giataganas P, Yang GZ. Implicit active constraints for robot-assisted arthroscopy. IEEE International Conference on Robotics and Automation (ICRA) 2013: 5390–5395

  31. Dai Z, Wu Z, Zhao J, Xu K. A robotic laparoscopic tool with enhanced capabilities and modular actuation. Sci China Technol Sci 2019; 62(1): 47–59

    Google Scholar 

  32. Çavuşoğlu MC, Cohn M, Tendick F, Sastry S. A laparoscopic telesurgical workstation. IEEE Trans Robot Autom 1999; 15(4): 728–739

    Google Scholar 

  33. Sekimoto M, Nishikawa A, Taniguchi K, Takiguchi S, Miyazaki F, Doki Y, Mori M. Development of a compact laparoscope manipulator (P-arm). Surg Endosc 2009; 23(11): 2596–2604

    PubMed  Google Scholar 

  34. Dalvand MM, Shirinzadeh B. Remote centre-of-motion control algorithms of 6-RRCRR parallel robot assisted surgery system (PRAMiSS). IEEE International Conference on Robotics and Automation (ICRA) 2012: 3401–3406

  35. Çavuşoğlu MC, Williams W, Tendick F, Sastry SS. Robotics for telesurgery: second generation Berkeley/UCSF laparoscopic tele-surgical workstation and looking towards the future applications. Industrial Robot 2003; 30(1): 22–29

    Google Scholar 

  36. Nasseri MA, Gschirr P, Eder M, Nair S, Kobuch K, Maier M, Zapp D, Lohmann C, Knoll A. Virtual fixture control of a hybrid parallel-serial robot for assisting ophthalmic surgery: an experimental study. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB) 2014: 732–738

  37. Azimian H, Patel RV, Naish MD. On constrained manipulation in robotics-assisted minimally invasive surgery. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB) 2010: 650–655

  38. Sandoval J, Poisson G, Vieyres P. A new kinematic formulation of the RCM constraint for redundant torque-controlled robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017: 4576–4581

  39. Funda J, Taylor RH, Eldridge B, Gomory S, Gruben KG. Constrained cartesian motion control for teleoperated surgical robots. IEEE Trans Robot Autom 1996; 12(3): 453–465

    Google Scholar 

  40. Locke RCO, Patel RV. Optimal remote center-of-motion location for robotics-assisted minimally-invasive surgery. IEEE International Conference on Robotics and Automation (ICRA) 2007:1900–1905

  41. Aghakhani N, Geravand M, Shahriari N, Vendittelli M, Oriolo G. Task control with remote center of motion constraint for minimally invasive robotic surgery. IEEE International Conference on Robotics and Automation (ICRA) 2013: 5807–5812

  42. Michelin M, Poignet P, Dombre E. Dynamic task/posture decoupling for minimally invasive surgery motions: simulation results. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2004: 3625–3630

  43. Cha HJ, Yi BJ. Modeling of a constraint force at RCM point in a needle insertion task. IEEE International Conference on Mechatronics and Automation (ICMA) 2011: 2177–2182

  44. Taylor RH, Funda J, Eldridge B, Gomory S, Gruben K, LaRose D, Talamini M, Kavoussi L, Anderson J. A telerobotics assistant for laparoscopic surgery. IEEE Eng Med Biol Mag 1995; 14(3): 279–288

    Google Scholar 

  45. Bunter SE, Ghodoussi M. Transforming a surgical robot for human telesurgery. IEEE Trans Robot Autom 2003; 19(5): 818–824

    Google Scholar 

  46. Wang W, Li J, Wang S, Su H, Jiang X. System design and animal experiment study of a novel minimally invasive surgical robot. Int J Med Robot 2016; 12(1): 73–84

    PubMed  Google Scholar 

  47. Ikuta K, Daifu S, Hasegawa T, Higashikawa H. Hyper-finger for remote minimally invasive surgery in deep area. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2002; 2488: 173–181

    Google Scholar 

  48. Dachs GW, Peine WJ. A novel surgical robot design: minimizing the operating envelope within the sterile field. Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS) 2006: 1505–1508

  49. Nakamura Y, Matsui A, Saito T, Yoshimoto K. Shape-memoryalloy active forceps for laparoscopic surgery. IEEE International Conference on Robotics and Automation (ICRA) 1995: 2320–2327

  50. Shi ZY, Liu D, Wang TM. A shape memory alloy-actuated surgical instrument with compact volume. Int J Med Robot 2014; 10(4): 474–481

    PubMed  Google Scholar 

  51. Takahashi H, Warisawa S, Mitsuishi M, Arata J, Hashizume M. Development of high dexterity minimally invasive surgical system with augmented force feedback capability. IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB) 2006: 284–289

  52. Hong MB, Jo YH. Design of a novel 4-DOF wrist-type surgical instrument with enhanced rigidity and dexterity. IEEE/ASME Trans Mechatron 2014; 19(2): 500–511

    Google Scholar 

  53. Robinson G, Davies JB. Continuum robots—a state of the art. IEEE International Conference on Robotics and Automation (ICRA) 1999: 2849–2853

  54. Francis P, Eastwood KW, Bodani V, Price K, Upadhyaya K, Podolsky D, Azimian H, Looi T, Drake J. Miniaturized instruments for the da Vinci research kit: design and implementation of custom continuum tools. IEEE Robot Autom Mag 2017; 24(2): 24–33

    Google Scholar 

  55. Abbott DJ, Becke C, Rothstein RI, Peine WJ. Design of an endoluminal NOTES robotic system. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2007: 410–416

  56. Kaouk JH, Haber GP, Autorino R, Crouzet S, Ouzzane A, Flamand V, Villers A. A novel robotic system for single-port urologic surgery: first clinical investigation. Eur Urol 2014; 66(6): 1033–1043

    PubMed  Google Scholar 

  57. Xu K, Zhao J, Fu M. Development of the SJTU unfoldable robotic system (SURS) for single port laparoscopy. IEEE/ASME Trans Mechatron 2015; 20(5): 2133–2145

    Google Scholar 

  58. Arata J, Fujisawa Y, Nakadate R, Kiguchi K, Harada K, Mitsuishi M, Hashizume M. Compliant four degree-of-freedom manipulator with locally deformable elastic elements for minimally invasive surgery. IEEE International Conference on Robotics and Automation (ICRA) 2019: 2663–2669

  59. Tobushi H, Furuichi Y, Sakuragi T, Sugimoto Y. Bending fatigue properties of a superelastic thin tube and a high-elastic thin wire of TiNi alloy. Mater Trans 2009; 50(8): 2043–2049

    CAS  Google Scholar 

  60. Vaida C, Plitea N, Pisla D, Gherman B. Orientation module for surgical instruments—a systematical approach. Meccanica 2013; 48(1): 145–158

    Google Scholar 

  61. Kanno T, Haraguchi D, Yamamoto M, Tadano K, Kawashima K. A forceps manipulator with flexible 4-DOF mechanism for laparoscopic surgery. IEEE/ASME Trans Mechatron 2015; 20(3): 1170–1178

    Google Scholar 

  62. Sa Z, Chen Y, Li Q, Zhao B, Xu K. Kinematic optimization of a continuum surgical manipulator. IEEE International Conference on Robotics and Biomimetics (ROBIO) 2018: 2069–2074

  63. Wu Z, Li Q, Zhao J, Gao J, Xu K. Design of a modular continuum-articulated laparoscopic robotic tool with decoupled kinematics. IEEE Robot Autom Lett 2019; 4(4): 3545–3552

    Google Scholar 

  64. Morelli L, Guadagni S, Di Franco G, Palmeri M, Di Candio G, Mosca F. Da Vinci single site© surgical platform in clinical practice: a systematic review. Int J Med Robot 2016; 12(4): 724–734

    PubMed  Google Scholar 

  65. Choi H, Kwak HS, Lim YA, Kim HJ. Surgical robot for single-incision laparoscopic surgery. IEEE Trans Biomed Eng 2014; 61(9): 2458–2466

    PubMed  Google Scholar 

  66. Seeliger B, Diana M, Ruurda JP, Konstantinidis KM, Marescaux J, Swanström LL. Enabling single-site laparoscopy: the SPORT platform. Surg Endosc 2019; 33(11): 3696–3703

    PubMed  PubMed Central  Google Scholar 

  67. Walker AS, Steele SR. The future of robotic instruments in colon and rectal surgery. Semin Colon Rectal Surg 2016; 27(3): 144–149

    Google Scholar 

  68. Roh KS, Yoon S, Do Kwon Y, Shim Y, Kim YJ. Single-port surgical robot system with flexible surgical instruments. International Conference on Intelligent Robotics and Applications (ICIRA) 2015: 447–459

  69. Petroni G, Niccolini M, Menciassi A, Dario P, Cuschieri A. A novel intracorporeal assembling robotic system for single-port laparoscopic surgery. Surg Endosc 2013; 27(2): 665–670

    PubMed  Google Scholar 

  70. Wortman TD, Mondry JM, Farritor SM, Oleynikov D. Single-site colectomy with miniature in vivo robotic platform. IEEE Trans Biomed Eng 2013; 60(4): 926–929

    PubMed  Google Scholar 

  71. Yung KL, Cheung JLK, Chung SW, Singh S, Yeung CK. A singleport robotic platform for laparoscopic surgery with a large central channel for additional instrument. Ann Biomed Eng 2017; 45(9): 2211–2221

    CAS  PubMed  Google Scholar 

  72. Kobayashi Y, Sekiguchi Y, Noguchi T, Takahashi Y, Liu Q, Oguri S, Toyoda K, Uemura M, Ieiri S, Tomikawa M, Ohdaira T, Hashizume M, Fujie MG. Development of a robotic system with six-degrees-of-freedom robotic tool manipulators for single-port surgery. Int J Med Robot 2015; 11(2): 235–246

    PubMed  Google Scholar 

  73. Cheon B, Gezgin E, Ji DK, Tomikawa M, Hashizume M, Kim HJ, Hong J. A single port laparoscopic surgery robot with high force transmission and a large workspace. Surg Endosc 2014; 28(9): 2719–2729

    PubMed  Google Scholar 

  74. Xu K, Goldman RE, Ding J, Allen PK, Fowler DL, Simaan N. System design of an insertable robotic effector platform for single port access (SPA) surgery. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2009: 5546–5552

  75. Wang H, Zhang R, Chen W, Wang X, Pfeifer R. A cable-driven soft robot surgical system for cardiothoracic endoscopic surgery: preclinical tests in animals. Surg Endosc 2017; 31(8): 3152–3158

    PubMed  Google Scholar 

  76. Zhao J, Feng B, Zheng MH, Xu K. Surgical robots for SPL and NOTES: a review. Minim Invasive Ther Allied Technol 2015; 24(1): 8–17

    PubMed  Google Scholar 

  77. Mintz Y, Horgan S, Cullen J, Stuart D, Falor E, Talamini MA. NOTES: a review of the technical problems encountered and their solutions. J Laparoendosc Adv Surg Tech A 2008; 18(4): 583–587

    PubMed  Google Scholar 

  78. Ikuta K, Yamamoto K, Sasaki K. Development of remote microsurgery robot and new surgical procedure for deep and narrow space. IEEE International Conference on Robotics and Automation (ICRA) 2003: 1103–1108

  79. Phee SJ, Low SC, Sun ZL, Ho KY, Huang WM, Thant ZM. Robotic system for no-scar gastrointestinal surgery. Int J Med Robot 2008; 4(1): 15–22

    CAS  PubMed  Google Scholar 

  80. Lehman AC, Dumpert J, Wood NA, Redden L, Visty AQ, Farritor S, Varnell B, Oleynikov D. Natural orifice cholecystectomy using a miniature robot. Surg Endosc 2009; 23(2): 260–266

    PubMed  Google Scholar 

  81. Tortora G, Dario P, Menciassi A. Array of robots augmenting the kinematics of endocavitary surgery. IEEE/ASME Trans Mechatron 2014; 19(6): 1821–1829

    Google Scholar 

  82. Son J, Cho CN, Kim KG, Chang TY, Jung H, Kim SC, Kim MT, Yang N, Kim TY, Sohn DK. A novel semi-automatic snake robot for natural orifice transluminal endoscopic surgery: preclinical tests in animal and human cadaver models (with video). Surg Endosc 2015; 29(6): 1643–1647

    PubMed  Google Scholar 

  83. Clark J, Noonan DP, Vitiello V, Sodergren MH, Shang J, Payne CJ, Cundy TP, Yang GZ, Darzi A. A novel flexible hyperredundant surgical robot: prototype evaluation using a single incision flexible access pelvic application as a clinical exemplar. Surg Endosc 2015; 29(3): 658–667

    PubMed  Google Scholar 

  84. Zhao J, Zheng X, Zheng M, Shih AJ, Xu K. An endoscopic continuum testbed for finalizing system characteristics of a surgical robot for NOTES procedures. IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) 2013: 63–70

  85. Goldman RE, Bajo A, MacLachlan LS, Pickens R, Herrell SD, Simaan N. Design and performance evaluation of a minimally invasive telerobotic platform for transurethral surveillance and intervention. IEEE Trans Biomed Eng 2013; 60(4): 918–925

    PubMed  Google Scholar 

  86. Burgner J, Rucker DC, Gilbert HB, Swaney PJ, Russell PT 3rd, Weaver KD, Webster RJ 3rd. A telerobotic system for transnasal surgery. IEEE/ASME Trans Mechatron 2014; 19(3): 996–1006

    Google Scholar 

  87. Thakkar S, Awad M, Gurram KC, Tully S, Wright C, Sanan S, Choset H. A novel, new robotic platform for natural orifice distal pancreatectomy. Surg Innov 2015; 22(3): 274–282

    PubMed  Google Scholar 

  88. Mitchell CR, Hendrick RJ, Webster RJ 3rd, Herrell SD. Toward improving transurethral prostate surgery: development and initial experiments with a prototype concentric tube robotic platform. J Endourol 2016; 30(6): 692–696

    PubMed  PubMed Central  Google Scholar 

  89. Légner A, Diana M, Halvax P, Liu YY, Zorn L, Zanne P, Nageotte F, De Mathelin M, Dallemagne B, Marescaux J. Endoluminal surgical triangulation 2.0: a new flexible surgical robot. Preliminary pre-clinical results with colonic submucosal dissection. Int J Med Robot 2017; 13(3): e1819

    Google Scholar 

  90. Xu K, Liang B, Dai Z, Zhao J, Zhao B, Liu H, Xiao L, Sun Y. Preliminary development of a continuum dual-arm surgical robotic system for transurethral procedures. International Conference on Intelligent Robotics and Applications (ICIRA) 2017: 311–322

  91. Wagner CR, Howe RD. Force feedback benefit depends on experience in multiple degree of freedom robotic surgery task. IEEE Trans Robot Autom 2007; 23(6): 1235–1240

    Google Scholar 

  92. Wagner CR, Stylopoulos N, Howe RD. The role of force feedback in surgery: analysis of blunt dissection. Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS) 2002: 68–74

  93. Xu K, Simaan N. An investigation of the intrinsic force sensing capabilities of continuum robots. IEEE Trans Robot 2008; 24(3): 576–587

    Google Scholar 

  94. Hong MB, Jo YH. Design and evaluation of 2-DOF compliant forceps with force-sensing capability for minimally invasive robot surgery. IEEE Trans Robot 2012; 28(4): 932–941

    Google Scholar 

  95. Sokhanvar S, Packirisamy M, Dargahi J. A multifunctional PVDF-based tactile sensor for minimally invasive surgery. Smart Mater Struct 2007; 16(4): 989–998

    CAS  Google Scholar 

  96. Zarrin PS, Escoto A, Xu R, Patel RV, Naish MD, Trejos AL. Development of an optical fiber-based sensor for grasping and axial force sensing. IEEE International Conference on Robotics and Automation (ICRA) 2017: 939–944

  97. Kim U, Kim YB, So J, Seok D, Choi HR. Sensorized surgical forceps for robotic-assisted minimally invasive surgery. IEEE Trans Ind Electron 2018; 65(12): 9604–9613

    Google Scholar 

  98. He C, Wang S, Sang H, Li J, Zhang L. Force sensing of multiple-DOF cable-driven instruments for minimally invasive robotic surgery. Int J Med Robot 2014; 10(3): 314–324

    CAS  PubMed  Google Scholar 

  99. Lee D, Kim U, Gulrez T, Yoon WJ, Hannaford B, Choi HR. A laparoscopic grasping tool with force sensing capability. IEEE/ASME Trans Mechatron 2016; 21(1): 130–141

    Google Scholar 

  100. Tholey G, Desai JP, Castellanos AE. Force feedback plays a significant role in minimally invasive surgery results and analysis. Ann Surg 2005; 241(1): 102–109

    PubMed  PubMed Central  Google Scholar 

  101. Seibold U, Kubler B, Hirzinger G. Prototype of instrument for minimally invasive surgery with 6-axis force sensing capability. IEEE International Conference on Robotics and Automation (ICRA) 2005: 496–501

  102. Moradi Dalvand M, Shirinzadeh B, Shamdani AH, Smith J, Zhong Y. An actuated force feedback-enabled laparoscopic instrument for robotic-assisted surgery. Int J Med Robot 2014; 10(1): 11–21

    PubMed  Google Scholar 

  103. Peirs J, Clijnen J, Reynaerts D, Brussel HV, Herijgers P, Corteville B, Boone S. A micro optical force sensor for force feedback during minimally invasive robotic surgery. Sens Actuators A Phys 2004; 115(2): 447–455

    CAS  Google Scholar 

  104. Noh Y, Sareh S, Würdemann H, Liu H, Back J, Housden J, Rhode K, Althoefer K. Three-axis fiber-optic body force sensor for flexible manipulators. IEEE Sens J 2016; 16(6): 1641–1651

    Google Scholar 

  105. Haslinger R, Leyendecker P, Seibold U. A fiberoptic force-torque-sensor for minimally invasive robotic surgery. 2013 IEEE International Conference on Robotics and Automation 2013: 4390–4395

  106. Qasaimeh MA, Sokhanvar S, Dargahi J, Kahrizi M. PVDF-based microfabricated tactile sensor for minimally invasive surgery. J Microelectromech Syst 2009; 18(1): 195–207

    CAS  Google Scholar 

  107. Xu K, Simaan N. Intrinsic wrench estimation and its performance index of multi-segment continuum robots. IEEE Trans Robot 2010; 26(3): 555–561

    Google Scholar 

  108. Lee SJ, Lee SC, Ahn HS. Design and control of tele-matched surgery robot. Mechatronics 2014; 24(5): 395–406

    Google Scholar 

  109. Kuebler B, Seibold U, Hirzinger G. Development of actuated and sensor integrated forceps for minimally invasive robotic surger. Int J Med Robot 2005; 1(3): 96–107

    CAS  PubMed  Google Scholar 

  110. Baumhauer M, Feuerstein M, Meinzer HP, Rassweiler J. Navigation in endoscopic soft tissue surgery: perspectives and limitations. J Endourol 2008; 22(4): 751–766

    PubMed  Google Scholar 

  111. Schols RM, Bouvy ND, van Dam RM, Stassen LP. Advanced intraoperative imaging methods for laparoscopic anatomy navigation: an overview. Surg Endosc 2013; 27(6): 1851–1859

    PubMed  Google Scholar 

  112. Stoyanov D, Mylonas GP, Deligianni F, Darzi A, Yang GZ. Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2005: 139–146

  113. Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD. Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 2009; 73(4): 896–900

    PubMed  Google Scholar 

  114. Röhl S, Bodenstedt S, Suwelack S, Dillmann R, Speidel S, Kenngott H, Muller-Stich BP. Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. Med Phys 2012; 39(3): 1632–1645

    PubMed  Google Scholar 

  115. Clancy NT, Stoyanov D, Maier-Hein L, Groch A, Yang GZ, Elson DS. Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging. Biomed Opt Express 2011; 2(11): 3119–3128

    PubMed  PubMed Central  Google Scholar 

  116. Schmalz C, Forster F, Schick A, Angelopoulou E. An endoscopic 3D scanner based on structured light. Med Image Anal 2012; 16(5): 1063–1072

    PubMed  Google Scholar 

  117. Le HND, Nguyen H, Wang Z, Opfermann J, Leonard S, Krieger A, Kang JU. Demonstration of a laparoscopic structured-illumination three-dimensional imaging system for guiding reconstructive bowel anastomosis. J Biomed Opt 2018; 23(5): 1–10

    PubMed  Google Scholar 

  118. Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D. Optical techniques for 3D surface reconstruction in computerassisted laparoscopic surgery. Med Image Anal 2013; 17(8): 974–996

    CAS  PubMed  Google Scholar 

  119. Bartoli A, Gerard Y, Chadebecq F, Collins T. On template-based reconstruction from a single view: analytical solutions and proofs of wellposedness for developable, isometric and conformal surfaces. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2012: 2026–2033

  120. Wu C, Narasimhan SG, Jaramaz B. A multi-image shape-from-shading framework for near-lighting perspective endoscopes. Int J Comput Vis 2010; 86(2–3): 211–228

    Google Scholar 

  121. Mountney P, Stoyanov D, Davison A, Yang GZ. Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2006: 347–354

  122. Mountney P, Yang GZ. Motion compensated SLAM for image guided surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2010: 496–504

  123. Mahmoud N, Cirauqui I, Hostettler A, Doignon C, Soler L, Marescaux J, Montiel JMM. ORBSLAM-based endoscope tracking and 3D reconstruction. International Workshop on Computer-Assisted and Robotic Endoscopy (CARE) 2016: 72–83

  124. Song J, Wang J, Zhao L, Huang S, Dissanayake G. MIS-SLAM: real-time large-scale dense deformable SLAM system in minimal invasive surgery based on heterogeneous computing. IEEE Robot Autom Lett 2018; 3(4): 4068–4075

    Google Scholar 

  125. Daskalaki D, Aguilera F, Patton K, Giulianotti PC. Fluorescence in robotic surgery. J Surg Oncol 2015; 112(3): 250–256

    PubMed  Google Scholar 

  126. Regeling B, Thies B, Gerstner AOH, Westermann S, Müller NA, Bendix J, Laffers W. Hyperspectral imaging using flexible endoscopy for laryngeal cancer detection. Sensors (Basel) 2016; 16(8): E1288

    Google Scholar 

  127. Erden MS, Rosa B, Boularot N, Gayet B, Morel G, Szewczyk J. Conic-spiraleur: a miniature distal scanner for confocal micro-laparoscope. IEEE/ASME Trans Mechatron 2014; 19(6): 1786–1798

    Google Scholar 

  128. Frangioni JV. In vivo near-infrared fluorescence imaging. Curr Opin Chem Biol 2003; 7(5): 626–634

    CAS  PubMed  Google Scholar 

  129. Tobis S, Knopf J, Silvers C, Yao J, Rashid H, Wu G, Golijanin D. Near infrared fluorescence imaging with robotic assisted laparo-scopic partial nephrectomy: initial clinical experience for renal cortical tumors. J Urol 2011; 186(1): 47–52

    PubMed  Google Scholar 

  130. Bjurlin MA, Gan M, McClintock TR, Volpe A, Borofsky MS, Mottrie A, Stifelman MD. Near-infrared fluorescence imaging: emerging applications in robotic upper urinary tract surgery. Eur Urol 2014; 65(4): 793–801

    PubMed  Google Scholar 

  131. Spinoglio G, Priora F, Bianchi PP, Lucido FS, Licciardello A, Maglione V, Grosso F, Quarati R, Ravazzoni F, Lenti LM. Realtime near-infrared (NIR) fluorescent cholangiography in single-site robotic cholecystectomy (SSRC): a single-institutional prospective study. Surg Endosc 2013; 27(6): 2156–2162

    PubMed  Google Scholar 

  132. Wagner OJ, Louie BE, Vallières E, Aye RW, Farivar AS. Near-infrared fluorescence imaging can help identify the contralateral phrenic nerve during robotic thymectomy. Ann Thorac Surg 2012; 94(2): 622–625

    PubMed  Google Scholar 

  133. Holloway RW, Bravo RAM, Rakowski JA, James JA, Jeppson CN, Ingersoll SB, Ahmad S. Detection of sentinel lymph nodes in patients with endometrial cancer undergoing robotic-assisted staging: a comparison of colorimetric and fluorescence imaging. Gynecol Oncol 2012; 126(1): 25–29

    PubMed  Google Scholar 

  134. Jafari MD, Lee KH, Halabi WJ, Mills SD, Carmichael JC, Stamos MJ, Pigazzi A. The use of indocyanine green fluorescence to assess anastomotic perfusion during robotic assisted laparoscopic rectal surgery. Surg Endosc 2013; 27(8): 3003–3008

    PubMed  Google Scholar 

  135. Decker RS, Shademan A, Opfermann JD, Leonard S, Kim PCW, Krieger A. Biocompatible near-infrared three-dimensional tracking system. IEEE Trans Biomed Eng 2017; 64(3): 549–556

    PubMed  PubMed Central  Google Scholar 

  136. Li Q, He X, Wang Y, Liu H, Xu D, Guo F. Review of spectral imaging technology in biomedical engineering: achievements and challenges. J Biomed Opt 2013; 18(10): 100901

    PubMed  Google Scholar 

  137. Stoyanov D. Surgical vision. Ann Biomed Eng 2012; 40(2): 332–345

    PubMed  Google Scholar 

  138. Lian J, Zheng Y, Jiao W, Yan F, Zhao B. Deblurring sequential ocular images from multi-spectral imaging (MSI) via mutual information. Med Biol Eng Comput 2018; 56(6): 1107–1113

    PubMed  Google Scholar 

  139. Olweny EO, Faddegon S, Best SL, Jackson N, Wehner EF, Tan YK, Zuzak KJ, Cadeddu JA. Renal oxygenation during robotassisted laparoscopic partial nephrectomy: characterization using laparoscopic digital light processing hyperspectral imaging. J Endourol 2013; 27(3): 265–269

    PubMed  Google Scholar 

  140. Moccia S, Wirkert SJ, Kenngott H, Vemuri AS, Apitz M, Mayer B, De Momi E, Mattos LS, Maier-Hein L. Uncertainty-aware organ classification for surgical data science applications in laparoscopy. IEEE Trans Biomed Eng 2018; 65(11): 2649–2659

    PubMed  Google Scholar 

  141. Gmitro AF, Aziz D. Confocal microscopy through a fiber-optic imaging bundle. Opt Lett 1993; 18(8): 565–567

    CAS  PubMed  Google Scholar 

  142. Zuo S, Yang GZ. Endomicroscopy for computer and robot assisted intervention. IEEE Rev Biomed Eng 2017; 10: 12–25

    PubMed  Google Scholar 

  143. Zuo S, Hughes M, Seneci C, Chang TP, Yang GZ. Toward intraoperative breast endomicroscopy with a novel surface-scanning device. IEEE Trans Biomed Eng 2015; 62(12): 2941–2952

    PubMed  Google Scholar 

  144. Giataganas P, Hughes M, Yang GZ. Force adaptive robotically assisted endomicroscopy for intraoperative tumour identification. Int J CARS 2015; 10(6): 825–832

    Google Scholar 

  145. Rosa B, Erden MS, Vercauteren T, Herman B, Szewczyk J, Morel G. Building large mosaics of confocal edomicroscopic images using visual servoing. IEEE Trans Biomed Eng 2013; 60(4): 1041–1049

    PubMed  Google Scholar 

  146. Giataganas P, Vitiello V, Simaiaki V, Lopez E, Yang GZ. Cooperative in situ microscopic scanning and simultaneous tissue surface reconstruction using a compliant robotic manipulator. IEEE International Conference on Robotics and Automation (ICRA) 2013: 5378–5383

  147. Mountney P, Giannarou S, Elson D, Yang GZ. Optical biopsy mapping for minimally invasive cancer screening. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2009: 483–490

  148. Avgousti S, Christoforou EG, Panayides AS, Voskarides S, Novales C, Nouaille L, Pattichis CS, Vieyres P. Medical telerobotic systems: current status and future trends. Biomed Eng Online 2016; 15(1): 96

    PubMed  PubMed Central  Google Scholar 

  149. Yang GZ, Cambias J, Cleary K, Daimler E, Drake J, Dupont PE, Hata N, Kazanzides P, Martel S, Patel RV, Santos VJ, Taylor RH. Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy. Sci Robot 2017; 2(4): eaam8638

    Google Scholar 

  150. Yip M, Das N. Robot autonomy for surgery. The Encyclopedia of Medical Robotics. World Scientific Publishing, 2018: 281–313

  151. Rovetta A, Sala R, Cosmi F, Wen X, Sabbadini D, Milanesi S, Togno A, Angelini L, Bejczy A. The first experiment in the world of robotic telesurgery for laparoscopy carried out by means of satellites networks and optical fibres networks on 7th July 1993. International Conference on Industrial Electronics, Control, and Instrumentation (IECON) 1993; 1: 51–56

    Google Scholar 

  152. Okamura AM. Methods for haptic feedback in teleoperated robotassisted surgery. Ind Rob 2004; 31(6): 499–508

    CAS  PubMed  PubMed Central  Google Scholar 

  153. Talasaz A, Trejos AL, Patel RV. Effect of force feedback on performance of robotics-assisted suturing. IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) 2012: 823–828

  154. Birglen L, Gosselin C, Pouliot N, Monsarrat B, Laliberté T. SHaDe, a new 3-DOF haptic device. IEEE Trans Robot Autom 2002; 18(2): 166–175

    Google Scholar 

  155. Steger R, Lin K, Adelstein BD, Kazerooni H. Design of a passively balanced spatial linkage haptic interface. J Mech Des 2004; 126(6): 984–991

    Google Scholar 

  156. Arata J, Kondo H, Ikedo N, Fujimoto H. Haptic device using a newly developed redundant parallel mechanism. IEEE Trans Robot 2011; 27(2): 201–214

    Google Scholar 

  157. Tholey G, Desai JPA. General-purpose 7 DOF haptic device: applications toward robot-assisted surgery. IEEE/ASME Trans Mechatron 2007; 12(6): 662–669

    Google Scholar 

  158. Stocco LJ, Salcudean SE, Sassani F. Optimal kinematic design of a haptic pen. IEEE/ASME Trans Mechatron 2001; 6(3): 210–220

    Google Scholar 

  159. Ueberle M, Buss M. Design, control, and evaluation of a new 6 DOF haptic device. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2002: 2949–2954

  160. Kim K, Kyun W. Design and analysis of a new 7-DOF parallel type haptic device: PATHOS-II. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2003: 2241–2246

  161. Faulring EL, Colgate JE, Peshkin MA. The cobotic hand controller: design, control and performance of a novel haptic display. Int J Robot Res 2006; 25(11): 1099–1119

    Google Scholar 

  162. Najdovski Z, Nahavandi S, Fukuda T. Design, development, and evaluation of a pinch-grasp haptic interface. IEEE/ASME Trans Mechatron 2014; 19(1): 45–54

    Google Scholar 

  163. Lee G, Hur SM, Oh Y. A novel haptic device with high-force display capability and wide workspace. IEEE International Conference on Robotics and Automation (ICRA) 2016: 2704–2709

  164. Zhao B, Sa Z, Wu Z, Li Q, Xu K. CombX: design of a haptic device for teleoperation. IEEE International Conference on Robotics and Biomimetics (ROBIO) 2018: 1969–1974

  165. Hokayem PF, Spong MW. Bilateral teleoperation: an historical survey. Automatica 2006; 42(12): 2035–2057

    Google Scholar 

  166. Hashtrudi-Zaad K, Salcudean SE. Analysis of control architectures for teleoperation systems with impedance/admittance master and slave manipulators. Int J Robot Res 2001; 20(6): 419–445

    Google Scholar 

  167. Lawrence DA. Stability and transparency in bilateral teleoperation. IEEE Trans Robot Autom 1993; 9(5): 624–637

    Google Scholar 

  168. Tavakoli M, Aziminejad A, Patel RV, Moallem M. High-fidelity bilateral teleoperation systems and the effect of multimodal haptics. IEEE Trans Syst Man Cybern B Cybern 2007; 37(6): 1512–1528

    PubMed  Google Scholar 

  169. Mahvash M, Okamura A. Friction compensation for enhancing transparency of a teleoperator with compliant transmission. IEEE Trans Robot 2007; 23(6): 1240–1246

    PubMed  PubMed Central  Google Scholar 

  170. Torabi A, Khadem M, Zareinia K, Sutherland GR, Tavakoli M. Application of a redundant haptic interface in enhancing soft-tissue stiffness discrimination. IEEE Robot Autom Lett 2019; 4(2): 1037–1044

    Google Scholar 

  171. Preusche C, Ortmaier T, Hirzinger G. Teleoperation concepts in minimal invasive surgery. Control Eng Pract 2002; 10(11): 1245–1250

    Google Scholar 

  172. Tavakoli M, Patel RV, Moallem M. Haptic interaction in robotassisted endoscopic surgery: a sensorized end-effector. Int J Med Robot 2005; 1(2): 53–63

    CAS  PubMed  Google Scholar 

  173. Kim U, Seok DY, Kim YB, Lee DH, Choi HR. Development of a grasping force-feedback user interface for surgical robot system. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2016: 845–850

  174. Cavusoglu MC, Sherman A, Tendick F. Design of bilateral teleoperation controllers for haptic exploration and telemanipulation of soft environments. IEEE Trans Robot Autom 2002; 18(4): 641–647

    Google Scholar 

  175. Tavakoli M, Howe RD. Haptic effects of surgical teleoperator flexibility. Int J Robot Res 2009; 28(10): 1289–1302

    Google Scholar 

  176. Rosenberg LB. Virtual fixtures: perceptual tools for telerobotic manipulation. Proceedings of IEEE Virtual Reality Annual International Symposium 1993: 76–82

  177. Abbott JJ, Marayong P, Okamura AM. Haptic virtual fixtures for robot-assisted manipulation. Robotics Research 2007; 28: 49–64

    Google Scholar 

  178. Lopez E, Zollo L, Guglielmelli E. Teleoperated control based on virtual fixtures for a redundant surgical system. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013: 450–455

  179. Kapoor A, Li M, Taylor RH. Spatial motion constraints for robot assisted suturing using virtual fixtures. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2005: 89–96

  180. Shahbazi M, Atashzar SF, Patel RV. A dual-user teleoperated system with virtual fixtures for robotic surgical training. IEEE International Conference on Robotics and Automation (ICRA) 2013: 3639–3644

  181. Vitrani MA, Poquet C, Morel G. Applying virtual fixtures to the distal end of a minimally invasive surgery instrument. IEEE Trans Robot 2017; 33(1): 114–123

    Google Scholar 

  182. Chen Z, Malpani A, Chalasani P, Deguet A, Vedula SS, Kazanzides P, Taylor RH. Virtual fixture assistance for needle passing and knot tying. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2016: 2343–2350

  183. Fontanelli GA, Yang GZ, Siciliano B. A comparison of assistive methods for suturing in MIRS. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2018: 4389–4395

  184. Haidegger T. Autonomy for surgical robots: concepts and paradigms. IEEE Transactions on Medical Robotics and Bionics 2019; 1(2): 65–76

    Google Scholar 

  185. Moustris GP, Hiridis SC, Deliparaschos KM, Konstantinidis KM. Evolution of autonomous and semi-autonomous robotic surgical systems: a review of the literature. Int J Med Robot 2011; 7(4): 375–392

    CAS  PubMed  Google Scholar 

  186. Hoeckelmann M, Rudas IJ, Fiorini P, Kirchne F, Haidegger T. Current capabilities and development potential in surgical robotics. Int J Adv Robot Syst 2015; 12(5): 61

    Google Scholar 

  187. Münzer B, Schoeffmann K, Böszörmenyi L. Content-based processing and analysis of endoscopic images and videos: a survey. Multimedia Tools Appl 2018; 77(1): 1323–1362

    Google Scholar 

  188. Wei GQ, Arbter K, Hirzinger G. Real-time visual servoing for laparoscopic surgery. Controlling robot motion with color image segmentation. IEEE Eng Med Biol Mag 1997; 16(1): 40–45

    CAS  PubMed  Google Scholar 

  189. Voros S, Long JA, Cinquin P. Automatic detection of instruments in laparoscopic images: a first step towards high-level command of robotic endoscopic holders. Int J Robot Res 2007; 26(11–12): 1173–1190

    Google Scholar 

  190. Pezzementi Z, Voros S, Hager GD. Articulated object tracking by rendering consistent appearance parts. IEEE International Conference on Robotics and Automation (ICRA) 2009: 3940–3947

  191. Kranzfelder M, Schneider A, Fiolka A, Schwan E, Gillen S, Wilhelm D, Schirren R, Reiser S, Jensen B, Feussner H. Real-time instrument detection in minimally invasive surgery using radio-frequency identification technology. J Surg Res 2013; 185(2): 704–710

    PubMed  Google Scholar 

  192. Krupa A, Gangloff J, Doignon C, de Mathelin MF, Morel G, Leroy J, Soler L, Marescaux J. Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE Trans Robot Autom 2003; 19(5): 842–853

    Google Scholar 

  193. Nageotte F, Zanne P, Doignon C, de Mathelin M. Visual servoing-based endoscopic path following for robot-assisted laparoscopic surgery. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2006: 2364–2369

  194. Reiter A, Allen PK, Zhao T. Feature classification for tracking articulated surgical tools. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2012: 592–600

  195. Ye M, Zhang L, Giannarou S, Yang GZ. Real-time 3D tracking of articulated tools for robotic surgery. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2016: 386–394

  196. Wolf R, Duchateau J, Cinquin P, Voros S. 3D tracking of laparoscopic instruments using statistical and geometric modeling. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2011: 203–210

  197. Allan M, Ourselin S, Thompson S, Hawkes DJ, Kelly J, Stoyanov D. Toward detection and localization of instruments in minimally invasive surgery. IEEE Trans Biomed Eng 2013; 60(4): 1050–1058

    PubMed  Google Scholar 

  198. Reiter A, Allen PK, Zhao T. Appearance learning for 3D tracking of robotic surgical tools. Int J Robot Res 2014; 33(2): 342–356

    Google Scholar 

  199. Reiter A, Goldman RE, Bajo A, Iliopoulos K, Simaan N, Allen PK. A learning algorithm for visual pose estimation of continuum robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2011: 2390–2396

  200. García-Peraza-Herrera LC, Li W, Fidon L, Gruijthuijsen C, Devreker A, Attilakos G, Deprest J, Poorten EV, Stoyanov D, Vercauteren T, Ourselin S. Toolnet: holistically-nested real-time segmentation of robotic surgical tools. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017: 5717–5722

  201. Shvets AA, Rakhlin A, Kalinin AA, Iglovikov VI. Automatic instrument segmentation in robot-assisted surgery using deep learning. IEEE International Conference on Machine Learning and Applications (ICMLA) 2018: 624–628

  202. Shademan A, Decker RS, Opfermann JD, Leonard S, Krieger A, Kim PCW. Supervised autonomous robotic soft tissue surgery. Sci Transl Med 2016; 8(337): 337ra64

    PubMed  Google Scholar 

  203. Nakamoto M, Ukimura O, Gill IS, Mahadevan A, Miki T, Hashizume M, Sato Y. Realtime organ tracking for endoscopic augmented reality visualization using miniature wireless magnetic tracker. International Workshop on Medical Imaging and Virtual Reality 2008: 359–366

  204. Nosrati MS, Peyrat JM, Abinahed J, Al-Alao O, Al-Ansari A, Abugharbieh R, Hamarneh G. Efficient multi-organ segmentation in multi-view endoscopic videos using pre-operative priors. Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2014: 324–331

  205. Bilodeau GA, Shu Y, Cheriet F. Multistage graph-based segmentation of thoracoscopic images. Comput Med Imaging Graph 2006; 30(8): 437–446

    PubMed  Google Scholar 

  206. Tjoa MP, Krishnan SM, Kugean C, Wang P, Doraiswaini R. Segmentation of clinical endoscopic image based on homogeneity and hue. International Conference of the IEEE Engineering in Medicine and Biology Society 2001: 2665–2668

  207. Figueiredo IN, Moreno JC, Prasath VBS, Figueiredo PN. A segmentation model and application to endoscopic images. International Conference Image Analysis and Recognition 2012: 164–171

  208. Wu S, Nakao M, Matsuda T. Continuous lung region segmentation from endoscopic images for intra-operative navigation. Comput Biol Med 2017; 87(1): 200–210

    PubMed  Google Scholar 

  209. Bodenstedt S, Wagner M, Mayer B, Stemmer K, Kenngott H, Müller-Stich B, Dillmann R, Speidel S. Image-based laparoscopic bowel measurement. Int J CARS 2016; 11(3): 407–419

    Google Scholar 

  210. Chhatkuli A, Bartoli A, Malti A, Collins T. Live image parsing in uterine laparoscopy. International Symposium on Biomedical Imaging 2014: 1263–1266

  211. Moccia S, Foti S, Rossi SM, Rota I, Scotti M, Toffoli S, Mattos LS, Momi ED, Frontoni E. FCNN-based segmentation of kidney vessels—towards constraints definition for safe robot-assisted nephrectomy. Joint Workshop on New Technologies for Computer/Robot Assisted Surgery 2018

  212. Rosen J, Hannaford B, Richards CG, Sinanan MN. Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills. IEEE Trans Biomed Eng 2001; 48(5): 579–591

    CAS  PubMed  Google Scholar 

  213. Lin HC, Shafran I, Yuh D, Hager GD. Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg 2006; 11(5): 220–230

    PubMed  Google Scholar 

  214. Murali A, Sen S, Kehoe B, Garg A, McFarland S, Patil S, Boyd WD, Lim S, Abbeel P, Goldberg K. Learning by observation for surgical subtasks: multilateral cutting of 3D viscoelastic and 2D orthotropic tissue phantoms. IEEE International Conference on Robotics and Automation (ICRA) 2015: 1202–1209

  215. Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N. Statistical modeling and recognition of surgical workflow. Med Image Anal 2012; 16(3): 632–641

    PubMed  Google Scholar 

  216. Ahmidi N, Tao L, Sefati S, Gao Y, Lea C, Haro BB, Zappella L, Khudanpur S, Vidal R, Hager GD. A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE Trans Biomed Eng 2017; 64(9): 2025–2041

    PubMed  PubMed Central  Google Scholar 

  217. Tao L, Elhamifar E, Khudanpur S, Hager GD, Vidal R. Sparse hidden Markov models for surgical gesture classification and skill evaluation. International Conference on Information Processing in Computer-Assisted Interventions 2012: 167–177

  218. Zappella L, Béjar B, Hager G, Vidal R. Surgical gesture classification from video and kinematic data. Med Image Anal 2013; 17(7): 732–745

    PubMed  Google Scholar 

  219. Despinoy F, Bouget D, Forestier G, Penet C, Zemiti N, Poignet P, Jannin P. Unsupervised trajectory segmentation for surgical gesture recognition in robotic training. IEEE Trans Biomed Eng 2016; 63(6): 1280–1291

    PubMed  Google Scholar 

  220. Murali A, Garg A, Krishnan S, Pokorny FT, Abbeel P, Darrell T, Goldberg K. TSC-DL: unsupervised trajectory segmentation of multi-modal surgical demonstrations with deep learning. IEEE International Conference on Robotics and Automation (ICRA) 2016: 4150–4157

  221. Krishnan S, Garg A, Patil S, Lea C, Hager G, Abbeel P, Goldberg K. Transition state clustering: unsupervised surgical trajectory segmentation for robot learning. Int J Robot Res 2017; 36(13–14): 1595–1618

    Google Scholar 

  222. Kang H, Wen JT. Endobot: a robotic assistant in minimally invasive surgeries. IEEE Int Conf Robot Autom 2001; 2: 2031–2036 (ICRA)

    Google Scholar 

  223. Nageotte F, Zanne P, Doignon C, de Mathelin M. Stitching planning in laparoscopic surgery: towards robot-assisted suturing. Int J Robot Res 2009; 28(10): 1303–1321

    Google Scholar 

  224. Jackson RC, Çavuşoğlu MC. Needle path planning for autonomous robotic surgical suturing. IEEE International Conference on Robotics and Automation (ICRA) 2013: 1669–1675

  225. Sen S, Garg A, Gealy DV, McKinley S, Jen Y, Goldberg K. Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization. IEEE International Conference on Robotics and Automation (ICRA) 2016: 4178–4185

  226. Staub C, Osa T, Knoll A, Bauernschmitt R. Automation of tissue piercing using circular needles and vision guidance for computer aided laparoscopic surgery. IEEE International Conference on Robotics and Automation (ICRA) 2010: 4585–4590

  227. Leonard S, Wu KL, Kim Y, Krieger A, Kim PCW. Smart tissue anastomosis robot (STAR): a vision-guided robotics system for laparoscopic suturing. IEEE Trans Biomed Eng 2014; 61(4): 1305–1317

    PubMed  Google Scholar 

  228. Zhong F, Wang Y, Wang Z, Liu YH. Dual-arm robotic needle insertion with active tissue deformation for autonomous suturing. IEEE Robot Autom Lett 2019; 4(3): 2669–2676

    Google Scholar 

  229. Mayer H, Gomez F, Wierstra D, Nagy I, Knoll A, Schmidhuber J. A system for robotic heart surgery that learns to tie knots using recurrent neural networks. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2006: 543–548

  230. Mayer H, Nagy I, Burschka D, Knoll A, Braun EU, Lange R, Bauernschmitt R. Automation of manual tasks for minimally invasive surgery. International Conference on Autonomic and Autonomous Systems (ICAS) 2008: 260–265

  231. Jvd B. Miller S, Duckworth D, Hu H, Wan A, Fu X-Y, Goldberg K, Abbeel P. Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations. IEEE International Conference on Robotics and Automation (ICRA) 2010: 2074–2081

  232. Schulman J, Gupta A, Venkatesan S, Tayson-Frederick M, Abbeel P. A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2013: 4111–4117

  233. Kehoe B, Kahn G, Mahler J, Kim J, Lee A, Lee A, Nakagawa K, Patil S, Boyd WD, Abbeel P, Goldberg K. Autonomous multilateral debridement with the raven surgical robot. IEEE International Conference on Robotics and Automation (ICRA) 2014: 1432–1439

  234. Le HND, Opfermann JD, Kam M, Raghunathan S, Saeidi H, Leonard S, Kang JU, Krieger A. Semi-autonomous laparoscopic robotic electro-surgery with a novel 3D Endoscope. IEEE International Conference on Robotics and Automation (ICRA) 2018: 6637–6644

  235. Nichols KA, Okamura AM. Autonomous robotic palpation: machine learning techniques to identify hard inclusions in soft tissues. IEEE International Conference on Robotics and Automation (ICRA) 2013: 4384–4389

  236. Nagy DÁ, Nagy TD, Elek R, Rudas IJ, Haidegger T. Ontology-based surgical subtask automation, automating blunt dissection. J Med Robot Res 2018; 3(03n04): 1841005

    Google Scholar 

  237. Nagy TD, Haidegger T. A DVRK-based framework for surgical subtask automation. Acta Polytech Hung 2019; 16(8): 61–78

    Google Scholar 

  238. Awtar S, Trutna TT, Nielsen JM, Abani R, Geiger J. FlexDex™:a minimally invasive surgical tool with enhanced dexterity and intuitive control. J Med Device 2010; 4(3): 035003–035011

    Google Scholar 

  239. Miyazaki R, Hirose K, Ishikawa Y, Kanno T, Kawashima K. A master-slave integrated surgical robot with active motion transformation using wrist axis. IEEE/ASME Trans Mechatron 2018; 23(3): 1215–1225

    Google Scholar 

  240. Roche ET, Horvath MA, Wamala I, Alazmani A, Song S-E, Whyte W, Machaidze Z, Payne CJ, Weaver JC, Fishbein G, Kuebler J, Vasilyev NV, Mooney DJ, Pigula FA, Walsh CJ. Soft robotic sleeve supports heart function. Sci Transl Med 2017; 9(373): eaaf3925

    PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Nos. 51722507, 51435010 and 91648103), and in part by the National Key R&D Program of China (No. 2017YFC0110800).

Author information

Authors and Affiliations

  1. State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Kai Xu

  2. RII Lab (Lab of Robotics Innovation and Intervention), UM-SJTU Joint Institute, Shanghai Jiao Tong University, Shanghai, 200240, China

    Yuyang Chen, Shu’an Zhang & Zhonghao Wu

  3. Department of Urology, Shanghai Changhai Hospital, the Second Military Medical University, Shanghai, 200433, China

    Bo Yang

  4. Shanghai Lung Cancer Center, Shanghai Chest Hospital, Shanghai Jiao Tong University, Shanghai, 200030, China

    Qingquan Luo

Authors
  1. Yuyang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shu’an Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhonghao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bo Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qingquan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kai Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai Xu.

Ethics declarations

Yuyang Chen, Shu’an Zhang, Zhonghao Wu, Bo Yang, Qingquan Luo, and Kai Xu declare that they have no financial conflicts of interest. This manuscript is a review article and does not involve a research protocol requiring approval by a relevant institutional review board or ethics committee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Zhang, S., Wu, Z. et al. Review of surgical robotic systems for keyhole and endoscopic procedures: state of the art and perspectives. Front. Med. 14, 382–403 (2020). https://doi.org/10.1007/s11684-020-0781-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11684-020-0781-x

Keywords

Search

Navigation