Skip to main content
SpringerLink
Log in

A Scaled Bilateral Teleoperation System for Robotic-Assisted Surgery with Time Delay

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The master-slave teleoperated robotic systems have advanced the surgeries in the past decades. Time delay is usually caused due to the data transmission between communication channel connecting the master and slave in bilateral teleoperation, which is crucial because even small time delay could destabilize the whole teleoperation system. Motivated to solve the instability caused by time delay in bilateral teleoperation, wave variable transformation (WVT) structure has been proposed to passivate the delayed communication channel. However, conventional WVT structure provides poor velocity, position and force tracking performances which are not sufficient for surgical applications. In this paper, a new wave variable compensation (WVC) structure is proposed to improve the tracking performances with less conservative condition and comprehensive analysis to keep stable and improved tracking performance is also provided. In order to better facilitate certain surgical procedures with special requirements, e.g. robotic-assisted neurosurgery, velocity/position and force scalings are designed in the proposed structure with guaranteed system passivity, and transparency of the scaled WVC structure is also analyzed. Simulation and experimental studies were carried out to verify the performance of the proposed structure with time delay. System performance comparisons with several existing wave based bilateral teleoperation structures are also provided through simulation studies to show the improvements brought by the proposed teleoperation structure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Nguan, C., Miller, B., Patel, R., Luke, P.P., Schlachta, C.M.: Pre-clinical remote telesurgery trial of a da vinci telesurgery prototype. Int. J. Med. Rob. Comput. Assisted Surg. 4(4), 304–309 (2008)

    Article  Google Scholar 

  2. Panait, L., Akkary, E., Bell, R.L., Roberts, K.E., Dudrick, S.J., Duffy, A.J.: The role of haptic feedback in laparoscopic simulation training. J. Surg. Res. 156(2), 312–316 (2009)

    Article  Google Scholar 

  3. Tahmasebi, A.M., Hashtrudi-Zaad, K., Thompson, D., Abolmaesumi, P.: A framework for the design of a novel haptic-based medical training simulator. IEEE Trans. Inf. Technol. Biomed. 12(5), 658–666 (2008)

    Article  Google Scholar 

  4. Artigas, J., Hirzinger, G.: A brief history of dlr’s space telerobotics and force-feedback teleoperation. Acta Polytech. Hung. 13(1), 239–249 (2016)

    Google Scholar 

  5. Marturi, N., Rastegarpanah, A., Takahashi, C., Adjigble, M., Stolkin, R., Zurek, S., Kopicki, M., Talha, M., Kuo, J.A., Bekiroglu, Y.: Towards advanced robotic manipulation for nuclear decommissioning: a pilot study on tele-operation and autonomy. In: 2016 International Conference on Robotics and Automation for Humanitarian Applications (RAHA), pp. 1–8. IEEE (2016)

  6. Murphy, R.R., Dreger, K.L., Newsome, S., Rodocker, J., Steimle, E., Kimura, T., Makabe, K., Matsuno, F., Tadokoro, S., Kon, K.: Use of remotely operated marine vehicles at minamisanriku and rikuzentakata Japan for disaster recovery. In: 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 19–25. IEEE (2011)

  7. Hokayem, P.F., Spong, M.W.: Bilateral teleoperation: an historical survey. Automatica 42(12), 2035–2057 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  8. Li, Z., Xia, Y., Sun, F.: Adaptive fuzzy control for multilateral cooperative teleoperation of multiple robotic manipulators under random Network-Induced delays. IEEE Trans. Fuzzy Systems 22(2), 437–450 (2014)

    Article  Google Scholar 

  9. Li, Z., Xia, Y., Wang, D., Zhai, D., Su, C.Y., Zhao, X.: Neural Network-Based control of networked trilateral teleoperation with geometrically unknown constraints. IEEE Trans. Cybernetics 46(5), 1051–1064 (2016)

    Article  Google Scholar 

  10. Bergeles, C., Yang, G.-Z.: From passive tool holders to microsurgeons: safer, smaller, smarter surgical robots. IEEE Trans. Biomed. Eng. 61(5), 1565–1576 (2014)

    Article  Google Scholar 

  11. Vitiello, V., Lee, S.-L., Cundy, T.P., Yang, G.-Z.: Emerging robotic platforms for minimally invasive surgery. IEEE Rev. Biomed. Eng. 6, 111–126 (2013)

    Article  Google Scholar 

  12. Taylor, R.H., Menciassi, A., Fichtinger, G., Fiorini, P., Dario, P.: Medical robotics and computer-integrated surgery. In: Springer Handbook of Robotics, pp. 1657–1684. Springer (2016)

  13. Karas, C.S., Chiocca, E.A.: Neurosurgical robotics: a review of brain and spine applications. J. Robot. Surg. 1(1), 39–43 (2007)

    Article  Google Scholar 

  14. Burgner, J., Rucker, D.C., Gilbert, H.B., Swaney, P.J., Russell, P.T., Weaver, K.D., Webster, R.J.: A telerobotic system for transnasal surgery. IEEE/ASME Trans. Mechatron. 19(3), 996–1006 (2014)

    Article  Google Scholar 

  15. Mohr, F.W., Falk, V., Diegeler, A., Walther, T., Gummert, J.F., Bucerius, J., Jacobs, S., Autschbach, R.: Computer-enhanced robotic cardiac surgery: experience in 148 patients. J. Thorac. Cardiovasc. Surg. 121(5), 842–853 (2001)

    Article  Google Scholar 

  16. Okamura, A.M.: Methods for haptic feedback in teleoperated robot-assisted surgery. Industrial Robot: An International Journal 31(6), 499–508 (2004)

    Article  Google Scholar 

  17. Tavakoli, M.: Haptics for teleoperated surgical robotic systems. World Scientific Publishing Co. Inc., Singapore (2008)

    Book  Google Scholar 

  18. Christopher, R., Nicholas, S., Robert, D.: The role of force feedback in surgery: analysis of blunt dissection. In: 10Th Symposium on Haptic Interface for Virtual Environment and Teleoperator Systems, vol. 1, pp 18–125. IEEE Computer Society, Orlando (2002)

  19. Meli, L., Pacchierotti, C., Prattichizzo, D.: Sensory subtraction in robot-assisted surgery: fingertip skin deformation feedback to ensure safety and improve transparency in bimanual haptic interaction. IEEE Trans. Biomed. Eng. 61(4), 1318–1327 (2014)

    Article  Google Scholar 

  20. ANSI-AAMI-ST79, Comprehensive guide to steam sterilization and sterility assurance in health care facilities. In: A1:2010, A2:2011, Association for the Advancement of Medical Instrumentation Arlington, VA

  21. Konstantinova, J., Jiang, A., Althoefer, K., Dasgupta, P., Nanayakkara, T.: Implementation of tactile sensing for palpation in robot-assisted minimally invasive surgery: a review. IEEE Sens. J 14(8), 2490–2501 (2014)

    Article  Google Scholar 

  22. Hinterseer, P., Hirche, S., Chaudhuri, S., Steinbach, E., Buss, M.: Perception-based data reduction and transmission of haptic data in telepresence and teleaction systems. IEEE Trans. Signal Process. 56(2), 588–597 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  23. Kuschel, M., Kremer, P., Buss, M.: Passive haptic data-compression methods with perceptual coding for bilateral presence systems. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 39(6), 1142–1151 (2009)

    Article  Google Scholar 

  24. Kokkonis, G., Psannis, K., Roumeliotis, M.s, Kontogiannis, S.: A survey of transport protocols for haptic applications. In: 2012 16th Panhellenic Conference on Informatics (PCI), pp. 192–197. IEEE (2012)

  25. Tortora, G., Dario, P., Menciassi, A.: Array of robots augmenting the kinematics of endocavitary surgery. IEEE/ASME Trans. Mechatron. 19(6), 1821–1829 (2014)

    Article  Google Scholar 

  26. Guo, J., Liu, C., Poignet, P.: Scaled position-force tracking for wireless teleoperation of miniaturized surgical robotic system. In: Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, pp. 361–365. IEEE (2014)

  27. Shine, T.S.J., et al.: Specialized Operating Room (Chapter 13), Operation Room Design Manual, pp 44–56. American Society of Anesthesiologists, Illinois, USA (2012)

    Google Scholar 

  28. Anderson, R.J., Spong, M.W.: Bilateral control of teleoperators with time delay. IEEE Trans. Autom. Control 34(5), 494–501 (1989)

    Article  MathSciNet  Google Scholar 

  29. Niemeyer, G., Slotine, J.-J.: Stable adaptive teleoperation. IEEE J. Ocean. Eng. 16(1), 152–162 (1991)

    Article  Google Scholar 

  30. Baier, H., Schmidt, G.: Transparency and stability of bilateral kinesthetic teleoperation with time-delayed communication. J. Intell. Robot. Syst. 40(1), 1–22 (2004)

    Article  Google Scholar 

  31. Nuño, E., Basañez, L., Ortega, R.: Passivity-based control for bilateral teleoperation: a tutorial. Automatica 47(3), 485–495 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  32. Deng, Q.-W., Wei, Q., Li, Z.-X.: Analysis of absolute stability for time-delay teleoperation systems. Int. J. Autom. Comput. 4(2), 203–207 (2007)

    Article  Google Scholar 

  33. Franken, M., Stramigioli, S., Misra, S., Secchi, C., Macchelli, A.: Bilateral telemanipulation with time delays: a two-layer approach combining passivity and transparency. IEEE Trans. Robot. 27(4), 741–756 (2011)

    Article  Google Scholar 

  34. Yang, C., Wang, X., Li, Z., Li, Y., Su, C.-Y.: Teleoperation control based on combination of wave variable and neural networks. IEEE Trans. Syst. Man Cybern. Syst. 47(8), 2125–2136 (2017)

    Article  Google Scholar 

  35. Kawashima, K., Tadano, K., Wang, C., Sankaranarayanan, G., Hannaford, B.: Bilateral teleoperation with time delay using modified wave variable based controller. In: 2009. ICRA’09 IEEE International Conference on Robotics and Automation, pp. 4326-4331. IEEE (2009)

  36. Bate, L., Cook, C.D., Li, Z.: Reducing wave-based teleoperator reflections for unknown environments. IEEE Trans. Ind. Electron. 2(58), 392–397 (2011)

    Article  Google Scholar 

  37. Ye, Y., Liu, P.X.: Improving haptic feedback fidelity in wave-variable-based teleoperation orientated to telemedical applications. IEEE Trans. Instrum. Meas. 58(8), 2847–2855 (2009)

    Article  Google Scholar 

  38. Ye, Y., Liu, P.X.: Improving trajectory tracking in wave-variable-based teleoperation. IEEE/ASME Trans. Mechatron. 15(2), 321–326 (2010)

    Article  Google Scholar 

  39. Li, H., Kawashima, K.: Achieving stable tracking in wave-variable-based teleoperation. IEEE/ASME Trans. Mechatron. 19(5), 1574–1582 (2014)

    Article  Google Scholar 

  40. Zhu, J., He, X., Gueaieb, W.: Trends in the control schemes for bilateral teleoperation with time delay. In: Kamel, M., Karray, F., Gueaieb, W., Khamis, A. (eds.) Autonomous and Intelligent Systems. Lecture Notes in Computer Science, vol. 6752. Springer, Berlin (2011)

  41. Muradore, R., Fiorini, P.: A review of bilateral teleoperation algorithms. Acta Polytechnica Hungarica 13 (1), 191–208 (2016)

    Google Scholar 

  42. Ghavifekr, A.A., Ghiasi, A.R., Badamchizadeh, M.A.: Discrete-time control of bilateral teleoperation systems: a review. Robotica 36(4), 552–569 (2018)

    Article  Google Scholar 

  43. Guo, J., Liu, C., Poignet, P.: Stable and enhanced position-force tracking for bilateral teleoperation with time delay. In: Control Conference (ECC), 2015 European, pp. 1980-1985. IEEE (2015)

  44. Lawrence, D.A.: Stability and transparency in bilateral teleoperation. IEEE Trans. Robot. Autom. 9(5), 624–637 (1993)

    Article  Google Scholar 

  45. Munir, S., Book, W.J.: Wave-based teleoperation with prediction. In: American Control Conference, 2001. Proceedings of the 2001, vol. 6, pp. 4605–4611. IEEE (2001)

  46. Niemeyer, G.D.: Using wave variables in time delayed force reflecting teleoperation, Ph.D. thesis Massachusetts Institute of Technology (1996)

  47. Ching, H., Book, W.J.: Internet-based bilateral teleoperation based on wave variable with adaptive predictor and direct drift control. J. Dyn. Syst. Meas. Control. 128(1), 86–93 (2006)

    Article  Google Scholar 

  48. Moreira, P., Zemiti, N., Liu, C., Poignet, P.: Viscoelastic model based force control for soft tissue interaction and its application in physiological motion compensation. Comput. Methods Prog. Biomed. 116(2), 52–67 (2014)

    Article  Google Scholar 

  49. Sanchez, L.A., Le, M., Liu, C., Zemiti, N., Poignet, P.: The impact of interaction model on stability and transparency in bilateral teleoperation for medical applications. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1607–1613. IEEE (2012)

  50. Hashtrudi-Zaad, K., Salcudean, S.E.: Analysis of control architectures for teleoperation systems with impedance/admittance master and slave manipulators. Int. J. Robot. Res. 20(6), 419–445 (2001)

    Article  Google Scholar 

  51. Downing, S.W., Herzog, W.A., McLaughlin, J.S., Gilbert, T.P.: Beating-heart mitral valve surgery: preliminary model and methodology. J. Thorac. Cardiovasc. Surg. 123(6), 1141–1146 (2002)

    Article  Google Scholar 

  52. Mitsuishi, M.: Medical robot and master slave system for minimally invasive surgery. In: 2007. CME 2007. IEEE/ICME International Conference on Complex Medical Engineering, pp. 8–13. IEEE (2007)

  53. Puvanesarajah, V., Liauw, J.A., Lo, S.-f., Lina, I.A., Witham, T.F.: Techniques and accuracy of thoracolumbar pedicle screw placement. World J. Orthod. 5(2), 112 (2014)

    Article  Google Scholar 

  54. Tavakoli, M., Patel, R.V., Moallem, M.: Bilateral control of a teleoperator for soft tissue palpation: design and experiments. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006. pp. 3280-3285. IEEE (2006)

  55. Christiansson, G.A., Van Der Helm, F.C.: The low-stiffness teleoperator slave—a trade-off between stability and performance. Int. J. Robot. Res. 26(3), 287–299 (2007)

    Article  Google Scholar 

  56. Colonnese, N., Okamura, A.M.: Analysis of effective impedance transmitted to the operator in position-exchange bilateral teleoperation. In: World Haptics Conference (WHC), pp. 328–333. IEEE (2017)

  57. Moreira, P., Patil, S., Alterovitz, R., Misra, S.: Needle steering in biological tissue using ultrasound-based online curvature estimation. In: Proceedings of IEEE Int Conf Robot Autom, pp. 4368–4373. Hong Kong (2014)

  58. Jian, B., Gao, W., Kacher, D., Nevo, E., Fetics, B., Lee, T.C., Jayender, J.: Kalman filter-based EM-optical sensor fusion for needle deflection estimation. Int. J. Comput. Assist. Radiol. Surg. 13(4), 573–583 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by Guangzhou Elite Project and National Natural Science Foundation of China (Grant 61803103).

Author information

Authors and Affiliations

  1. School of Automation, Guangdong University of Technology, Guangzhou, 510006, China

    Jing Guo

  2. LIRMM, CNRS-University of Montpellier, 161 Rue Ada, Montpellier, 34095, France

    Chao Liu & Philippe Poignet

Authors
  1. Jing Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philippe Poignet
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao Liu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, J., Liu, C. & Poignet, P. A Scaled Bilateral Teleoperation System for Robotic-Assisted Surgery with Time Delay. J Intell Robot Syst 95, 165–192 (2019). https://doi.org/10.1007/s10846-018-0918-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-018-0918-1

Keywords

Search

Navigation