Skip to main content
SpringerLink
Log in

Force-guided autonomous robotic ultrasound scanning control method for soft uncertain environment

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Autonomous ultrasound imaging by robotic ultrasound scanning systems in complex soft uncertain clinical environments is important and challenging to assist in therapy. To cope with the complex environment faced by the ultrasound probe during the scanning process, we propose an autonomous robotic ultrasound (US) control method based on reinforcement learning (RL) model to build the relationship between the environment and the system. The proposed method requires only contact force as input information to achieve robot control of the posture and contact force of the probe without any a priori information about the target and the environment.

Methods

First, an RL agent is proposed and trained by a policy gradient theorem-based RL model with the 6-degree-of-freedom (DOF) contact force of the US probe to learn the relationship between contact force and output force directly. Then, a force control strategy based on the admittance controller is proposed for synchronous force, orientation and position control by defining the desired contact force as the action space.

Results

The proposed method was evaluated via collected US images, contact force and scan trajectories by scanning an unknown soft phantom. The experimental results indicated that the proposed method differs from the free-hand scanned approach in the US images within 3 ± 0.4%. The analysis results of contact forces and trajectories indicated that our method could make stable scanning processes on a soft uncertain skin surface and obtained US images.

Conclusion

We propose a concise and efficient force-guided US robot scanning control method for soft uncertain environment based on reinforcement learning. Experimental results validated our method's feasibility and validity for complex skin surface scanning, and the volunteer experiments indicated the potential application value in the complex clinical environment of robotic US imaging system especially with limited visual information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Priester A, Natarajan S, Culjat M (2013) Robotic ultrasound systems in medicine. IEEE Trans Ultrason Ferroelect Freq Contr 60:507–523. https://doi.org/10.1109/TUFFC.2013.2593

    Article  Google Scholar 

  2. Hoeckelmann M, Rudas IJ, Fiorini P, Kirchner F, Haidegger T (2015) Current capabilities and development potential in surgical robotics. Int J Adv Robot Syst 12:61. https://doi.org/10.5772/60133

    Article  Google Scholar 

  3. Li G, Su H, Cole GA, Shang W, Harrington K, Camilo A, Pilitsis JG, Fischer GS (2015) Robotic system for MRI-guided stereotactic neurosurgery. IEEE Trans Biomed Eng 62:1077–1088. https://doi.org/10.1109/TBME.2014.2367233

    Article  PubMed  PubMed Central  Google Scholar 

  4. Lau KC, Leung EYY, Chiu PWY, Yam Y, Lau JYW, Poon CCY (2016) A flexible surgical robotic system for removal of early-stage gastrointestinal cancers by endoscopic submucosal dissection. IEEE Trans Ind Inf 12:2365–2374. https://doi.org/10.1109/TII.2016.2576960

    Article  Google Scholar 

  5. Huang Q, Zeng Z (2017) A review on real-time 3D ultrasound imaging technology. Biomed Res Int 2017:1–20. https://doi.org/10.1155/2017/6027029

    Article  Google Scholar 

  6. Kojcev R, Khakzar A, Fuerst B, Zettinig O, Fahkry C, DeJong R, Richmon J, Taylor R, Sinibaldi E, Navab N (2017) On the reproducibility of expert-operated and robotic ultrasound acquisitions. Int J CARS 12:1003–1011. https://doi.org/10.1007/s11548-017-1561-1

    Article  Google Scholar 

  7. Pan Z, Tian S, Guo M, Zhang J, Yu N, Xin Y (2017) Comparison of medical image 3D reconstruction rendering methods for robot-assisted surgery. In: IEEE 2017 2nd international conference on advanced robotics and mechatronics-ICARM 2019, vol 6, p 94. https://doi.org/10.1109/ICARM.2017.8273141

  8. Koizumi N, Joonho S, Deukhee L, Nomiya A, Yoshinaka K, Sugita N, Matsumoto Y, Homma Y, Mitsuishi M (2010) Integration of diagnostics and therapy by ultrasound and robot technology. In: IEEE 2010 international symposium on micro-nanomechatronics and human science -MHS 2010, pp 53–58. https://doi.org/10.1109/MHS.2010.5669577

  9. Boezaart A, Ihnatsenka B (2010) Ultrasound: basic understanding and learning the language. Int J Shoulder Surg 4(3):55. https://doi.org/10.4103/0973-6042.76960

    Article  PubMed  PubMed Central  Google Scholar 

  10. Chatelain P, Krupa A, Navab N (2017) Confidence-driven control of an ultrasound probe. IEEE Trans Robot 33(6):1410–1424. https://doi.org/10.1109/TRO.2017.2723618

    Article  Google Scholar 

  11. Haidegger T (2019) Autonomy for surgical robots: concepts and paradigms. IEEE Trans Med Robot Bionics 1(2):65–76. https://doi.org/10.1109/TMRB.2019.2913282

    Article  Google Scholar 

  12. Elek R, Nagy TD, Nagy DA, Takacs B, Galambos P, Rudas I, Haidegger T (2017) Robotic platforms for ultrasound diagnostics and treatment. IEEE Trans Syst Man Cybern. https://doi.org/10.1109/SMC.2017.8122869

    Article  Google Scholar 

  13. Hennersperger C, Fuerst B, Virga S, Zettinig O, Frisch B, Neff T, Navab N (2017) Towards MRI-based autonomous robotic US acquisitions: a first feasibility study. IEEE Trans Med Imaging 36(2):11. https://doi.org/10.1109/tmi.2016.2620723

    Article  Google Scholar 

  14. Huang Q, Wu B, Lan J, Li X (2018) Fully automatic three-dimensional ultrasound imaging based on conventional B-Scan. IEEE Trans Biomed Circuits Syst 12(2):426–436. https://doi.org/10.1109/TBCAS.2017.2782815

    Article  PubMed  Google Scholar 

  15. Huang Q, Lan J, Li X (2019) Robotic arm based automatic ultrasound scanning for three-dimensional imaging. IEEE Trans Industr Inform 15(2):10. https://doi.org/10.1109/tii.2018.2871864

    Article  Google Scholar 

  16. Hogan N (1985) Impedance control: an approach to manipulation. J Dyn Syst Meas Control 107(1):1–24

    Article  Google Scholar 

  17. Raibert MH, Craig JJ (1981) Hybrid position/force control of robot manipulators. J Dyn Syst Meas Control 103(2):126–133

    Article  Google Scholar 

  18. Gullapalli V, Grupen RA (1992) Learning reactive admittance control. In: IEEE 1992 international conference on robotics and automation-ICRA 1992, pp 1475–1480. https://doi.org/10.1109/robot.1992.220143

  19. Bruyninckx H, De Schutter J (1996) Specification of force-controlled actions in the “task frame formalism”—a synthesis. IEEE Trans Robot Automat 12(4):581–589. https://doi.org/10.1109/70.508440

    Article  Google Scholar 

  20. Duan J, Gan Y, Chen M, Dai X (2018) Adaptive variable impedance control for dynamic contact force tracking in uncertain environment. Robot Auton Syst 102:54–65. https://doi.org/10.1016/j.robot.2018.01.009

    Article  Google Scholar 

  21. Kiguchi K, Fukuda T (2000) Position/force control of robot manipulators for geometrically unknown objects using fuzzy neural networks. IEEE Trans Ind Electron 47(3):641–649. https://doi.org/10.1109/41.847905

    Article  Google Scholar 

  22. Baigzadehnoe B, Rahmani Z, Khosravi A, Rezaie B (2017) On position/force tracking control problem of cooperative robot manipulators using adaptive fuzzy backstepping approach. ISA Trans 70:432–446. https://doi.org/10.1016/j.isatra.2017.07.029

    Article  PubMed  Google Scholar 

  23. Kronander K, Billard A (2012) Online learning of varying stiffness through physical human–robot interaction. In: IEEE 2012 international conference on robotics and automation-ICRA 2012, pp 1842–1849. https://doi.org/10.1109/icra.2012.6224877

  24. Calinon S, Sardellitti I, Caldwell DG (2010) Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies. In: IEEE 2010 IEEE/RSJ international conference on intelligent robots and systems-IROS 2010, pp 249–254. https://doi.org/10.1109/IROS.2010.5648931

  25. Martín-Martín R, Lee MA, Gardner R, Savarese S, Bohg J, Garg A (2019) Variable impedance control in end-effector space: an action space for reinforcement learning in contact-rich tasks. In: IEEE 2019 IEEE/RSJ international conference on intelligent robots and systems-IROS 2019, pp 1010–1017. https://doi.org/10.1109/iros40897.2019.8968201

  26. Bence T, Tamás H (2020) Autonomous applied robotics: ultrasound-based robot-assisted needle insertion system concept and development. In: IEEE 2020 15th international conference of system of systems engineering-SoSE 2020, pp 487–492. https://doi.org/10.1109/SoSE50414.2020.9130499

  27. Grace C, Nickolas E, Anton D, Peter K (2019) A compliance model to improve the accuracy of the da Vinci Research Kit (dVRK). Acta Polytech Hung 16(8):49–60. https://doi.org/10.12700/APH.16.8.2019.8.4

    Article  Google Scholar 

  28. Yoshikawa T (1986) Dynamic hybrid position/force control of robot manipulators description of hand constraints and calculation of joint driving force. In: IEEE 1986 IEEE International conference on robotics and automation-ICRA1986, pp 1393–1398. https://doi.org/10.1109/ROBOT.1986.1087420.

  29. Nguyen TT, Nguyen ND, Nahavandi S (2020) Deep reinforcement learning for multiagent systems: a review of challenges, solutions, and applications. IEEE Trans Cybern 50(9):3826–3839. https://doi.org/10.1109/TCYB.2020.2977374

    Article  PubMed  Google Scholar 

  30. Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. In: https://arxiv.org/abs/1707.06347

Download references

Acknowledgements

This work was supported in part by National Natural Science Foundation of China (82027807, 81771940), Beijing Municipal Natural Science Foundation (7212202).

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China

    Guochen Ning, Jiaqi Chen, Xinran Zhang & Hongen Liao

Authors
  1. Guochen Ning
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiaqi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinran Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongen Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongen Liao.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

This article does contain a study with human participants and was approved by the Tsinghua University Institutional Review Board under the number 20210074. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (MP4 4692 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ning, G., Chen, J., Zhang, X. et al. Force-guided autonomous robotic ultrasound scanning control method for soft uncertain environment. Int J CARS 16, 2189–2199 (2021). https://doi.org/10.1007/s11548-021-02462-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02462-6

Keywords

Search

Navigation