Annals of Biomedical Engineering

, Volume 46, Issue 9, pp 1385–1396 | Cite as

Automatic Robotic Steering of Flexible Needles from 3D Ultrasound Images in Phantoms and Ex Vivo Biological Tissue

  • Paul Mignon
  • Philippe Poignet
  • Jocelyne TroccazEmail author


Robotic control of needle bending aims at increasing the precision of percutaneous procedures. Ultrasound feedback is preferable for its clinical ease of use, cost and compactness but raises needle detection issues. In this paper, we propose a complete system dedicated to robotized guidance of a flexible needle under 3D ultrasound imaging. This system includes a medical robot dedicated to transperineal needle positioning and insertion, a rapid path planning for needle steering using bevel-tip needle natural curvature in tissue, and an ultrasound-based automatic needle detection algorithm. Since ultrasound-based automatic needle steering is often made difficult by the needle localization in biological tissue, we quantify the benefit of using flexible echogenic needles for robotized guidance under 3D ultrasound. The “echogenic” term refers to the etching of microstructures on the needle shaft. We prove that these structures improve needle visibility and detection robustness in ultrasound images. We finally present promising results when reaching targets using needle steering. The experiments were conducted with various needles in different media (synthetic phantoms and ex vivo biological tissue). For instance, with nitinol needles the mean accuracy is 1.2 mm (respectively 3.8 mm) in phantoms (resp. biological tissue).


Needle steering Needle detection 3D ultrasound Echogenic needle Ex-vivo tissue Robotics 



This work was partly supported by the French ANR within the “Investissements d’Avenir” program (Labex CAMI) under reference ANR-11-LABX-0004.

Supplementary material

Supplementary material 1 (MP4 49885 kb)


  1. 1.
    Abayazid, M., M. Kemp, and S. Misra. 3d flexible needle steering in soft-tissue phantoms using fiber bragg grating sensors. 2013 IEEE International Conference on Robotics and Automation (ICRA). IEEE, Karlsruhe, 2013.Google Scholar
  2. 2.
    Abayazid, M., P. Moreira, N. Shahriari, S. Patil, R. Alterovitz, and S. Misra. Ultrasound-guided three-dimensional needle steering in biological tissue with curved surfaces. Med. Eng. Phys. 37(1):145–150, 2015.CrossRefPubMedGoogle Scholar
  3. 3.
    Adebar, T. K., A. E. Fletcher, and A. M. Okamura. 3-D ultrasound-guided robotic needle steering in biological tissue. IEEE Trans. Biomed. Eng. 61(12):2899–2910, 2014.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Adebar, T. K., and A. M. Okamura. Recursive estimation of needle pose for control of 3D-ultrasound-guided robotic needle steering. 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014). IEEE, Chicago, 2014.Google Scholar
  5. 5.
    Alterovitz, R., T. Siméon, and K. Y. Goldberg. The stochastic motion roadmap: a sampling framework for planning with Markov Motion Uncertainty. Robotics: Science and Systems. 3:233–241, 2007.Google Scholar
  6. 6.
    Beigi, P., R. Rohling, S. E. Salcudean, V. A. Lessoway, and G. C. Ng. Needle trajectory and tip localization in real-time 3-D ultrasound using a moving stylus. Ultrasound Med. Biol. 41(7):2057–2070, 2015.CrossRefPubMedGoogle Scholar
  7. 7.
    Bergin, D., J. N. Pappas, J. J. Hwang, D. H. Sheafor, and E. K. Paulson. Echogenic polymer coating: does it improve needle visualization in sonographically guided biopsy? Am. J. Roentgenol. 178(5):1188–1190, 2002.CrossRefGoogle Scholar
  8. 8.
    Bernardes, M., B. V. Adorno, P. Poignet, and G. Borges. Robot-assisted automatic insertion of steerable needles with closed-loop imaging feedback and intraoperative trajectory replanning. Mechatronics 23(6):630–645, 2013.CrossRefGoogle Scholar
  9. 9.
    Chatelain, P., A. Krupa, and N. Navab. 3D ultrasound-guided robotic steering of a flexible needle via visual servoing. 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2015.Google Scholar
  10. 10.
    Cheung, S., and R. Rohling. Enhancement of needle visibility in ultrasound-guided percutaneous procedures. Ultrasound Med. Biol. 30(5):617–624, 2004.CrossRefPubMedGoogle Scholar
  11. 11.
    DiMaio, S. P., and S. E. Salcudean. Needle steering and model-based trajectory planning. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Berlin, 2003.Google Scholar
  12. 12.
    Dupont, P. E., J. Lock, B. Itkowitz, and E. Butler. Design and control of concentric-tube robots. IEEE Trans. Robot. 26(2):209–225, 2010.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Glozman, D., and M. Shoham. Image-guided robotic flexible needle steering. IEEE Trans. Robot. 23(3):459–467, 2007.CrossRefGoogle Scholar
  14. 14.
    Hungr, N., M. Baumann, J.-A. Long, and J. Troccaz. A 3-D ultrasound robotic prostate brachytherapy system with prostate motion tracking. IEEE Trans. Robot. 28(6):1382–1397, 2012.CrossRefGoogle Scholar
  15. 15.
    Hungr, N., J.-A. Long, V. Beix, and J. Troccaz. A realistic deformable prostate phantom for multimodal imaging and needle-insertion procedures. Med. Phys. 39(4):2031–2041, 2012.CrossRefPubMedGoogle Scholar
  16. 16.
    Kamada, T., R. Yasumura, R. Takao, K. Suga, and Y. Aoyama. A quantitative comparative study of a new echogenic needle for nerve bocks. Anesthesiology 109:A344, 2008.Google Scholar
  17. 17.
    Ko, S. Y., L. Frasson, and F. Rodriguez y Baena. Closed-loop planar motion control of a steerable probe with a “programmable bevel” inspired by nature. IEEE Trans. Robot. 27(5):970–983, 2011.CrossRefGoogle Scholar
  18. 18.
    Mignon, P., P. Poignet, and J. Troccaz. Beveled-tip needle-steering using 3D ultrasound, mechanical-based Kalman filter and curvilinear ROI prediction. 2016 14th International Conference on Control, Automation, Robotics and Vision (ICARCV). IEEE, 2016.Google Scholar
  19. 19.
    Neubach, Z., and M. Shoham. Ultrasound-guided robot for flexible needle steering. IEEE Trans. Biomed. Eng. 57(4):799–805, 2010.CrossRefPubMedGoogle Scholar
  20. 20.
    Okazawa, S. H., R. Ebrahimi, J. Chuang, R. N. Rohling, and S. E. Salcudean. Methods for segmenting curved needles in ultrasound images. Med. Image Anal. 10(3):330–342, 2006.CrossRefPubMedGoogle Scholar
  21. 21.
    Park, J. W., M. W. Cheon, and M. H. Lee. Phantom study of a new laser-etched needle for improving visibility during ultrasonography-guided lumbar medial branch access with novices. Ann. Rehabil. Med. 40(4):575–582, 2016.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Park, W., J. S. Kim, Y. Zhou, N. J. Cowan, A. M. Okamura, and G. S. Chirikjian. Diffusion-based motion planning for a nonholonomic flexible needle model. Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. ICRA 2005. IEEE, 2005.Google Scholar
  23. 23.
    Patel, N. A., T. van Katwijk, G. Li, P. Moreira, W. Shang, S. Misra, and G. S. Fischer. Closed-loop asymmetric-tip needle steering under continuous intraoperative MRI guidance. New York: IEEE, 2015.CrossRefGoogle Scholar
  24. 24.
    Patil, S., J. Burgner, R. J. Webster, and R. Alterovitz. Needle steering in 3-D via rapid replanning. IEEE Trans. Robot. 30(4):853–864, 2014.CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Reed, K. B., A. M. Okamura, and N. J. Cowan. Modeling and control of needles with torsional friction. IEEE Trans. Biomed. Eng. 56(12):2905–2916, 2009.CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Rossa, C., and M. Tavakoli. Issues in closed-loop needle steering. Control Eng. Pract. 62:55–69, 2017.CrossRefGoogle Scholar
  27. 27.
    Rossa, C., N. Usmani, R. Sloboda, and M. Tavakoli. A Hand-Held Assistant for Semiautomated Percutaneous Needle Steering. IEEE Trans. Biomed. Eng. 64(3):637–648, 2017.CrossRefPubMedGoogle Scholar
  28. 28.
    Rucker, D. C., J. Das, H. B. Gilbert, P. J. Swaney, M. I. Miga, N. Sarkar, and R. J. Webster. Sliding mode control of steerable needles. IEEE Trans. Robot. 29(5):1289–1299, 2013.CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Strintzis, M. G., and I. Kokkinidis. Maximum likelihood motion estimation in ultrasound image sequences. IEEE Signal Process. Lett. 4(6):156–157, 1997.CrossRefGoogle Scholar
  30. 30.
    Tsumura, R., Y. Takishita, Y. Fukushima, and H. Iwata. Histological evaluation of tissue damage caused by rotational needle insertion. 2016 IEEE 38th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC). IEEE, 2016.Google Scholar
  31. 31.
    Uhercik, M., J. Kybic, H. Liebgott, and C. Cachard. Multi-resolution parallel integral projection for fast localization of a straight electrode in 3D ultrasound images. 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2008. ISBI 2008. IEEE, 2008.Google Scholar
  32. 32.
    Uhercik, M., J. Kybic, H. Liebgott, and C. Cachard. Model fitting using RANSAC for surgical tool localization in 3-D ultrasound images. IEEE Trans. Biomed. Eng. 57(8):1907–1916, 2010.CrossRefPubMedGoogle Scholar
  33. 33.
    van de Berg, N. J., D. J. van Gerwen, J. Dankelman, and J. J. van den Dobbelsteen. Design choices in needle steering—a review. IEEE/ASME Trans. Mechatron. 20(5):2172–2183, 2015.CrossRefGoogle Scholar
  34. 34.
    Webster, R. J., J. S. Kim, N. J. Cowan, G. S. Chirikjian, and A. M. Okamura. Nonholonomic modeling of needle steering. Int. J. Robot. Res. 25(5-6):509–525, 2006.CrossRefGoogle Scholar
  35. 35.
    Wedlick, T. R., and A. M. Okamura. Characterization of pre-curved needles for steering in tissue. Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009. EMBC 2009. IEEE, 2009.Google Scholar
  36. 36.
    Wood, N. A., K. Shahrour, M. C. Ost, and C. N. Riviere. Needle steering system using duty-cycled rotation for percutaneous kidney access. 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2010.Google Scholar
  37. 37.
    Wu, G., X. Li, C. A. Lehocky, C. N. Riviere, and G. Wu, et al. Automatic Steering of manually inserted needles. 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2013.Google Scholar
  38. 38.
    Xu, J., V. Duindam, R. Alterovitz, and K. Goldberg. Motion planning for steerable needles in 3D environments with obstacles using rapidly-exploring random trees and backchaining. IEEE International Conference on Automation Science and Engineering, 2008. CASE 2008. IEEE, 2008.Google Scholar

Copyright information

© Biomedical Engineering Society 2018

Authors and Affiliations

  1. 1.Université Grenoble Alpes, CNRS, Grenoble INP, TIMC-IMAGGrenobleFrance
  2. 2.LIRMM, Université de Montpellier, CNRSMontpellierFrance
  3. 3.TIMC-IMAG Laboratory, Pavillon Taillefer, School of MedicineLa Tronche CedexFrance

Personalised recommendations