Skip to main content
SpringerLink
Log in

An Integrated Sensor-Model Approach for Haptic Feedback of Flexible Endoscopic Robots

  • Published:
Annals of Biomedical Engineering Aims and scope Submit manuscript

Abstract

Haptic feedback for flexible endoscopic surgical robots is challenging due to space constraints for sensors and shape-dependent force hysteresis of tendon-sheath mechanisms (TSMs). This paper proposes (1) a single-axis fiber Bragg grating (FBG)-based force sensor for a TSM of a robotic arm and (2) an integrated sensor-model approach to estimate forces on other TSMs of that arm. With a robust and simple structure, a temperature-compensated sensor can be mounted on the distal sheath to measure forces applied by the TSM. This proposed sensor was integrated with a Ø4.2 mm articulated robotic arm driven by six TSMs, with a measurement error of 0.37 N in this work. The measurement from the single sensor was used to identify parameters in the force-transmission models of all other TSMs in the robot, realizing a one-sensor-for-all-distal-forces measurement method. The sensor-model approach could accurately estimate the distal force with an RMSE of 0.65 N. An animal study was carried out to demonstrate the sensor’s feasibility in real-life surgery. The sensor-model approach presented a robust, space-saving, and cost-effective solution for haptic feedback of endoscopic robots without any assumption on the shapes of the robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8

Similar content being viewed by others

References

  1. Agrawal, V., W. J. Peine, and B. Yao. Modeling of transmission characteristics across a cable-conduit system. IEEE Trans. Robot. 26(5):914–924, 2010.

    Article  Google Scholar 

  2. Arabagi, V., O. Felfoul, A. H. Gosline, R. J. Wood, and P. E. Dupont. Biocompatible pressure sensing skins for minimally invasive surgical instruments. IEEE Sens. J. 16(5):1294–1303, 2016.

    Article  CAS  Google Scholar 

  3. Cao, L., X. Li, P. T. Phan, A. M. H. Tiong, H. L. Kaan, K. Y. Ho, P. W. Y. Chiu, and S. J. Phee. A flexible endoscopic robotic suturing system for gastrointestinal perforations: animal study. In: Hamlyn Symposium on Medical Robotics, 2019.

  4. Cao, L., X. Li, P. Phan, A. Tiong, J. Liu, and S. J. Phee. A novel robotic suturing system for flexible endoscopic surgery. In: International Conference on Robotics and Automation. Montreal, Canada, 2019.

  5. Do, T. N., T. Tjahjowidodo, M. W. S. Lau, T. Yamamoto, and S. J. Phee. Hysteresis modeling and position control of tendon-sheath mechanism in flexible endoscopic systems. Mechatronics 24(1):12–22, 2014.

    Article  Google Scholar 

  6. Feng, J., Y. Yahui, Z. Zhao, et al. A variable-impedance piezoelectric tactile sensor with tunable sensing performance for tissue hardness sensing in robotic tumor palpation. Smart Mater. Struct. 27(11):115039, 2018.

    Article  Google Scholar 

  7. Gao, A., Y. Zhou, L. Cao, Z. Wang, and H. Liu. Fiber Bragg grating-based triaxial force sensor with parallel flexure hinges. IEEE Trans. Ind. Electron. 65(10):8215–8223, 2018.

    Article  Google Scholar 

  8. Gu, X., C. Li, X. Xiao, C. M. Lim, and H. Ren. A compliant transoral surgical robotic system based on a parallel flexible mechanism. Ann. Biomed. Eng. 2019. https://doi.org/10.1007/s10439-019-02241-0.

    Article  PubMed  PubMed Central  Google Scholar 

  9. He, X., J. Handa, P. Gehlbach, R. Taylor, and I. Iordachita. A submillimetric 3-DOF force sensing instrument with integrated fiber bragg grating for retinal microsurgery. IEEE Trans. Biomed. Eng. 61(2):522–534, 2014.

    Article  Google Scholar 

  10. Hill, K. O., and G. Meltz. Fiber Bragg grating technology fundamentals and overview. J. Lightwave Technol. 15(8):1263–1276, 1997.

    Article  CAS  Google Scholar 

  11. Kaneko, M., T. Yamashita, and K. Tanie. Basic considerations on transmission characteristics for tendon drive robots. Paper Presented at Advanced Robotics, 1991. Fifth International Conference on ‘Robots in Unstructured Environments’, 91 ICAR, 1991.

  12. Khoshnam, M., and R. V. Patel. Robotics-assisted control of steerable ablation catheters based on the analysis of tendon-sheath transmission mechanisms. IEEE/ASME Trans. Mechatron. 22(3):1473–1484, 2017.

    Article  Google Scholar 

  13. Kim, U., Y. B. Kim, D. Y. Seok, J. So, and H. R. Choi. A surgical palpation probe with 6-axis force/torque sensing capability for minimally invasive surgery. IEEE Trans. Ind. Electron. 65(3):2755–2765, 2018.

    Article  Google Scholar 

  14. Lai, W., L. Cao, Z. Xu, P. T. Phan, P. Shum, and S.J. Phee. Distal end force sensing with optical fiber Bragg gratings for tendon-sheath mechanisms in flexible endoscopic robots. Paper Presented at 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018.

  15. Le, H. M., T. N. Do, L. Cao, S.J. Phee. Towards active variable stiffness manipulators for surgical robots. Paper Presented at 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017.

  16. Li, X., L. Cao, A. M. H. Tiong, P. T. Phan, and S. J. Phee. Distal-end force prediction of tendon-sheath mechanisms for flexible endoscopic surgical robots using deep learning. Mech. Mach. Theory 134:323–337, 2019.

    Article  CAS  Google Scholar 

  17. Li, T., C. Shi, and H. Ren. A high-sensitivity tactile sensor array based on fiber Bragg grating sensing for tissue palpation in minimally invasive surgery. IEEE/ASME Trans. Mechatron. 2018. https://doi.org/10.1109/tmech.2018.2856897.

    Article  Google Scholar 

  18. Li, T., C. Shi, and H. Ren. Three-dimensional catheter distal force sensing for cardiac ablation based on fiber Bragg grating. IEEE/ASME Trans. Mechatron. 23(5):2316–2327, 2018.

    Article  Google Scholar 

  19. McKinley, S., A. Garg, and S. Sen, et al. A single-use haptic palpation probe for locating subcutaneous blood vessels in robot-assisted minimally invasive surgery. Paper Presented at 2015 IEEE International Conference on Automation Science and Engineering (CASE), 2015.

  20. Pacchierotti, C., D. Prattichizzo, and K. J. Kuchenbecker. Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery. IEEE Trans. Biomed. Eng. 63(2):278–287, 2016.

    Article  Google Scholar 

  21. Ren, F., W. Zhang, Y. Li, Y. Lan, Y. Xie, and W. Dai. The temperature compensation of FBG sensor for monitoring the stress on hole-edge. IEEE Photonics J. 10(4):1–9, 2018.

    Article  Google Scholar 

  22. Srivastava, A., R. Xu, A. Escoto, C. Ward, and R. V. Patel. Design of an ultra thin strain sensor using superelastic nitinol for applications in minimally invasive surgery. Paper Presented at 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), 2016.

  23. Su, H., W. Shang, G. Li, N. Patel, and G. S. Fischer. An MRI-guided telesurgery system using a Fabry-Perot interferometry force sensor and a pneumatic haptic device. Ann. Biomed. Eng. 45(8):1917–1928, 2017.

    Article  Google Scholar 

  24. Sun, Z., Z. Wang, and S. J. Phee. Elongation modeling and compensation for the flexible tendon-sheath system. IEEE/ASME Trans. Mechatron. 19(4):1243–1250, 2014.

    Article  Google Scholar 

  25. Wagner, C. R., N. Stylopoulos, P. G. Jackson, and R. D. Howe. The benefit of force feedback in surgery: examination of blunt dissection. Presence Teleoperators Virtual Environ. 16(3):252–262, 2007.

    Article  Google Scholar 

  26. Zarrin, P. S., A. Escoto, R. Xu, R. V. Patel, M. D. Naish, and A. L. Trejos. Development of a 2-DOF sensorized surgical grasper for grasping and axial force measurements. IEEE Sens. J. 18(7):2816–2826, 2018.

    Article  Google Scholar 

  27. Zhou, Z., and J. Ou. Techniques of temperature compensation for FBG strain sensors used in long-term structural monitoring. Paper Presented at Fundamental Problems of Optoelectronics and Microelectronics II, 2005.

Download references

Acknowledgments

This work was supported by the National Research Foundation (NRF), Singapore (NRFI2016-07).

Author information

Authors and Affiliations

  1. Robotics Research Center, School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore

    Wenjie Lai, Lin Cao, Xiaoguo Li, Phuoc Thien Phan, Anthony Meng Huat Tiong & Soo Jay Phee

  2. The Photonics Institute, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore

    Rex Xiao Tan & Swee Chuan Tjin

  3. Temasek Laboratories, Nanyang Technological University, Singapore, Singapore

    Yung Chuen Tan

Authors
  1. Wenjie Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rex Xiao Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yung Chuen Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoguo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Phuoc Thien Phan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Anthony Meng Huat Tiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Swee Chuan Tjin
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Soo Jay Phee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lin Cao.

Additional information

Associate Editor Tingrui Pan oversaw the review of this article.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lai, W., Cao, L., Tan, R.X. et al. An Integrated Sensor-Model Approach for Haptic Feedback of Flexible Endoscopic Robots. Ann Biomed Eng 48, 342–356 (2020). https://doi.org/10.1007/s10439-019-02352-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10439-019-02352-8

Keywords

Search

Navigation