Surgical Endoscopy

, Volume 26, Issue 8, pp 2117–2125 | Cite as

Review of surgical robotics user interface: what is the best way to control robotic surgery?

  • Anton Simorov
  • R. Stephen Otte
  • Courtni M. Kopietz
  • Dmitry Oleynikov



As surgical robots begin to occupy a larger place in operating rooms around the world, continued innovation is necessary to improve our outcomes.


A comprehensive review of current surgical robotic user interfaces was performed to describe the modern surgical platforms, identify the benefits, and address the issues of feedback and limitations of visualization.


Most robots currently used in surgery employ a master/slave relationship, with the surgeon seated at a work-console, manipulating the master system and visualizing the operation on a video screen. Although enormous strides have been made to advance current technology to the point of clinical use, limitations still exist. A lack of haptic feedback to the surgeon and the inability of the surgeon to be stationed at the operating table are the most notable examples. The future of robotic surgery sees a marked increase in the visualization technologies used in the operating room, as well as in the robots’ abilities to convey haptic feedback to the surgeon. This will allow unparalleled sensation for the surgeon and almost eliminate inadvertent tissue contact and injury.


A novel design for a user interface will allow the surgeon to have access to the patient bedside, remaining sterile throughout the procedure, employ a head-mounted three-dimensional visualization system, and allow the most intuitive master manipulation of the slave robot to date.


Surgical robot Minimally invasive surgery Surgical user interface 



Dmitry Oleynikov is a stockholder of Virtual Incision Corporation. R. Stephen Otte, Anton Simorov, and Courtni Kopietz have no conflicts of interest or financial ties to disclose.


  1. 1.
    Su L (2009) Role of robotics in modern urologic practice. Curr Opin Urol 19(1):63–64PubMedCrossRefGoogle Scholar
  2. 2.
    Lee DI (2009) Robotic prostatectomy: what we have learned and where we are going. Yonsei Med J 50:177–181. doi:10.3349/ymj.2009.50.2.177 PubMedCrossRefGoogle Scholar
  3. 3.
    Palep JH (2009) Robotic assisted minimally invasive surgery. J Minim Access Surg 5(1):1–7PubMedCrossRefGoogle Scholar
  4. 4.
    Shah A, Okotie OT, Zhao L, Pins MR, Bhalani V, Dalton DP (2008) Pathologic outcomes during the learning curve for robotic-assisted laparoscopic radical prostatectomy. Int Braz J Urol 34(2):159–163PubMedCrossRefGoogle Scholar
  5. 5.
    Tan GY, Goel RK, Kaouk JH, Tewari AK (2009) Technological advances in robotic-assisted laparoscopic surgery. Urol Clin North Am 36(2):237–249PubMedCrossRefGoogle Scholar
  6. 6.
    Phee SJ, Low SC, Huynh VA, Kencana AP, Sun ZL, Yang K (2009) Master and slave transluminal endoscopic robot (MASTER) for natural orifice transluminal endoscopic surgery (NOTES). Conf Proc IEEE Eng Med Biol Soc 2009:1192–1195PubMedGoogle Scholar
  7. 7.
    Sun Z, Ang RY, Lim EW, Wang Z, Ho KY, Phee SJ (2011) Enhancement of a master-slave robotic system for natural orifice transluminal endoscopic surgery. Ann Acad Med Singapore 40(5):223–228PubMedGoogle Scholar
  8. 8.
    Phee SJ, Ho KY, Lomanto D, Low SC, Huynh VA, Kencana AP, Yang K, Sun ZL, Chung SC (2010) Natural orifice transgastric endoscopic wedge hepatic resection in an experimental model using an intuitively controlled master and slave transluminal endoscopic robot (MASTER). Surg Endosc 24(9):2293–2298PubMedCrossRefGoogle Scholar
  9. 9.
    Ho KY, Phee SJ, Shabbir A, Low SC, Huynh VA, Kencana AP, Yang K, Lomanto D, So BY, Wong YY, Chung SC (2010) Endoscopic submucosal dissection of gastric lesions by using a Master and Slave Transluminal Endoscopic Robot (MASTER). Gastrointest Endosc 72(3):593–599PubMedCrossRefGoogle Scholar
  10. 10.
    Hagn U, Konietschke R, Tobergte A, Nickl M, Jörg S, Kübler B, Passig G, Gröger M, Fröhlich F, Seibold U, Le-Tien L, Albu-Schäffer A, Nothhelfer A, Hacker F, Grebenstein M, Hirzinger G (2010) DLR MiroSurge: a versatile system for research in endoscopic telesurgery. Int J Comput Assist Radiol Surg 5(2):183–193PubMedCrossRefGoogle Scholar
  11. 11.
    Kuebler B, Seibold U, Hirzinger G (2005) Development of actuated and sensor integrated forceps for minimally invasive robotic surgery. Int J Med Robot 1(3):96–107. doi:10.1581/mrcas.2005.010305and10.1002/rcs.33 PubMedCrossRefGoogle Scholar
  12. 12.
    Konietschke R, Hagn U, Nickl M, Jörg S, Tobergte A, Passig G, Seibold U, Le Tien L, Kuebler B, Gröger M, Fröhlich F, Rink C, Albu-Schäffer A, Grebenstein M, Ortmaier T, Hirzinger G (2009) The DLR MiroSurge: a robotic system for surgery. Video contribution presented at ICRAGoogle Scholar
  13. 13.
    Suppa M, Kielhofer S, Langwald J, Hacker F, Strobl KH, Hirzinger G (2007) The 3D-modeller: a multi-purpose vision platform. IEEE Int Conf Robot Autom 2007:781–787. doi:10.1109/ROBOT.2007.363081 Google Scholar
  14. 14.
    Harnett BM, Doarn CR, Rosen J, Hannaford B, Broderick TJ (2008) Evaluation of unmanned airborne vehicles and mobile robotic telesurgery in an extreme environment. Telemed J E Health 14(6):539–544PubMedCrossRefGoogle Scholar
  15. 15.
    Rosen J, Hannaford B (2006) Doc at a distance. IEEE Spectrum 6:34CrossRefGoogle Scholar
  16. 16.
    Bornhoft JM, Strabala KW, Wortman TD, Lehman AC, Oleynikov D, Farritor SM (2011) Stereoscopic visualization and haptic technology used to create a virtual environment for remote surgery. Biomed Sci Instrum 47:76–81PubMedGoogle Scholar
  17. 17.
    Zhang X, Nelson C, Oleynikov D (2011) Natural haptic interface for single-port surgical robot with gravity compensation. Int J Med Robot 7(S1). Epub 4 Nov 2011Google Scholar
  18. 18.
    Lum M, Friedman D, Rosen J et al (2009) The RAVEN–design and validation of a telesurgery system. Int J Rob Res 28(9):1183–1197CrossRefGoogle Scholar
  19. 19.
    Lang MJ, Greer AD, Sutherland GR (2011) Intra-operative robotics: NeuroArm. Acta Neurochir Suppl 109:231–236PubMedCrossRefGoogle Scholar
  20. 20.
    Samad MD, Hu Y, Sutherland GR (2010) Effect of force feedback from each DOF on the motion accuracy of a surgical tool in performing a robot-assisted tracing task. Conf Proc IEEE Eng Med Biol Soc 2010:2093–2096PubMedGoogle Scholar
  21. 21.
    Sutherland GR, Latour I, Greer AD (2008) Integrating an image-guided robot with intraoperative MRI. IEEE Eng Med Biol Mag 27(3):59–65PubMedCrossRefGoogle Scholar
  22. 22.
    Joel P, Rosen J, Burns S (2007) Upper-limb powered exoskeleton design. IEEE ASME Trans Mechatron 12(4):408–417CrossRefGoogle Scholar
  23. 23.
    Joel P, Powell J, Rosen J (2009) Isotropy of an upper limb exoskeleton and the kinematics and dynamics of the human arm. Appl Bionics Biomech 6(2):175–191CrossRefGoogle Scholar
  24. 24.
    Ettore C, Rosen J, Joel P, Burns S (2006) Myoprocessor for neural controlled powered exoskeleton arm. IEEE Trans Biomed Eng 53(11):2387–2396CrossRefGoogle Scholar
  25. 25.
    Wortman TD, Strabala KW, Lehman AC, Farritor SM, Oleynikov D (2011) Laparoendoscopic single-site surgery using a multi-functional miniature in vivo robot. Int J Med Robot 7(1):17–21PubMedCrossRefGoogle Scholar
  26. 26.
    Lehman A, Wood N, Farritor S, Goede M, Oleynikov D (2011) Dexterous miniature robot for advanced minimally invasive surgery. Surg Endosc 25(1):119–123PubMedCrossRefGoogle Scholar
  27. 27.
    Teber D, Baumhauer M, Guven EO et al (2009) Robotic and imaging in urological surgery. Curr Opin Urol 19:108–113PubMedCrossRefGoogle Scholar
  28. 28.
    Pamplona VF, Fernandes LAF, Prauchner J, Nedel LP, Olivier MM (2008) The image-based data glove. Proceedings of X Symposium on Virtual Real (SVR 2008) 204–211Google Scholar
  29. 29.
    Sturman DJ, Zeltzer D (1994) A survey of glove based-input. IEEE Comput Graph Appl 14(1):30–39CrossRefGoogle Scholar
  30. 30.
    Faraz A, Payandeh S (2000) Engineering approaches to mechanical and robotic design for minimally invasive surgery (MIS). Kluwer Academic Publishers, Boston, pp 1–11Google Scholar
  31. 31.
    Okamura AM (2009) Haptic feedback in robot-assisted minimally invasive surgery. Curr Opin Urol 19(1):102–107PubMedCrossRefGoogle Scholar
  32. 32.
    Tholey G, Desai JP (2007) A general purpose 7 DOF haptic device: applications towards robot-assisted surgery. IEEE/ASME Trans Mech 12(6):662–669CrossRefGoogle Scholar
  33. 33.
    Mavash M (2006) Novel approach for modeling separating forces between deformable bodies. IEEE Trans Inf Technol Biomed 10(3):618–626CrossRefGoogle Scholar
  34. 34.
    Mavash M, Hayward V (2004) High fidelity haptic synthesis of contact with deformable bodies. IEEE Comput Graph Appl 24(2):28–55Google Scholar
  35. 35.
    Mavash M, Voo LM, Kim D et al (2008) Modeling the forces of cutting with scissors. IEEE Trans Biomed Eng 55(3):848–856CrossRefGoogle Scholar
  36. 36.
    Weiss H, Ortmaier T, Maass H et al (2003) A virtual reality based haptic surgical training system. Comput Aided Surg 8(5):269–272PubMedCrossRefGoogle Scholar
  37. 37.
    Wagner CR, Howe RD (2007) Force feedback benefit depends on experience in multiple degree of freedom robotic surgery. IEEE Trans Rob 23(6):1235–1240CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Anton Simorov
    • 1
  • R. Stephen Otte
    • 1
  • Courtni M. Kopietz
    • 1
  • Dmitry Oleynikov
    • 1
  1. 1.University of Nebraska Medical Center, Center for Advanced Surgical TechnologyOmahaUSA

Personalised recommendations