Annals of Biomedical Engineering

, Volume 43, Issue 8, pp 1828–1840 | Cite as

Optical Flow-Based Tracking of Needles and Needle-Tip Localization Using Circular Hough Transform in Ultrasound Images



Image-guided interventions have become the standard of care for needle-based procedures. The success of the image-guided procedures depends on the ability to precisely locate and track the needle. This work is primarily focused on 2D ultrasound-based tracking of a hollow needle (cannula) that is composed of straight segments connected by shape memory alloy actuators. An in-plane tracking algorithm based on optical flow was proposed to track the cannula configuration in real-time. Optical flow is a robust tracking algorithm that can easily run on a CPU. However, the algorithm does not perform well when it is applied to the ultrasound images directly due to the intensity variation in the images. The method presented in this work enables using the optical flow algorithm on ultrasound images to track features of the needle. By taking advantage of the bevel tip, Circular Hough transform was used to accurately locate the needle tip when the imaging is out-of-plane. Through experiments inside tissue phantom and ex-vivo experiments in bovine kidney, the success of the proposed tracking methods were demonstrated. Using the methods presented in this work, quantitative information about the needle configuration is obtained in real-time which is crucial for generating control inputs for the needle and automating the needle insertion.


Ultrasound imaging Optical flow Hough transform Steerable cannula 


  1. 1.
    Aboofazeli, M., P. Abolmaesumi, P. Mousavi, and G. Fichtinger. A new scheme for curved needle segmentation in three-dimensional ultrasound images. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI), pp. 1067–1070, 2009.Google Scholar
  2. 2.
    Adebar, T. K., and A. Okamura. 3D segmentation of curved needles using doppler ultrasound and vibration. In: Information Processing in Computer-Assisted Interventions. Lecture Notes in Computer Science, vol. 7915, pp. 61–70, 2013.Google Scholar
  3. 3.
    Ayvali, E., and J. P. Desai. Towards a discretely actuated steerable cannula. In: 2012 IEEE International Conference on Robotics and Automation (ICRA) pp. 1614–1619, 2012.Google Scholar
  4. 4.
    Ayvali, E., C. Liang, M. Ho, Y. Chen, and J. P. Desai. Towards a discretely actuated steerable cannula for diagnostic and therapeutic procedures. Int. J. Robotics Res. 31:588–603, 2012.CrossRefGoogle Scholar
  5. 5.
    Bouguet, J. Y. Pyramidal Implementation of the Lucas Kanade Feature Tracker. Intel Corporation, Microprocessor Research Labs, 2000.Google Scholar
  6. 6.
    Bradski, G. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000.Google Scholar
  7. 7.
    Bradski, G., and A. Kaehler. Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media, 2008.Google Scholar
  8. 8.
    Chatelain, P., A. Krupa, and M. Marchal. Real-time needle detection and tracking using a visually servoed 3D ultrasound probe. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1676–1681, May 2013.Google Scholar
  9. 9.
    Chen, X., L. Lu, and Y. Gao. A new concentric circle detection method based on Hough transform. In: International Conference on Computer Science Education (ICCSE), pp. 753–758, 2012.Google Scholar
  10. 10.
    Deans, S. R. Hough transform from the radon transform. IEEE Trans. Pattern Anal. Mach. Intell. 3:185–188, 1981.Google Scholar
  11. 11.
    Ding, M., and A. Fenster. A real-time biopsy needle segmentation technique using Hough transform. Med. Phys. 30:2222–2233, 2003.PubMedCrossRefGoogle Scholar
  12. 12.
    Dong, B., E. Savitsky, and S. Osher. A novel method for enhanced needle localization using ultrasound-guidance. In: Advances in Visual Computing. Lecture Notes in Computer Science, vol. 5875. Berlin: Springer, pp. 914–923, 2009.Google Scholar
  13. 13.
    Duan, Q., K. M. Parker, A. Lorsakul, E. D. Angelini, E. Hyodo, J.W. Homma, S., Holmes, and A. F. Laine. Quantitative validation of optical flow based myocardial strain measures using sonomicrometry. In: Proceedings of IEEE International Symposium Biomedicin Imaging, vol. 2009, pp. 454–457, Jun 2009.Google Scholar
  14. 14.
    Duda, R. O., and P. E. Hart. Use of the Hough transformation to detect lines and curves in pictures. Commun. ACM 15:11–15, 1972.CrossRefGoogle Scholar
  15. 15.
    Fichtinger, G., J. Fiene, C. Kennedy, G. Kronreif, I. Iordachita, D. Song, E. C. Burdette, and P. Kazanzides. Robotic assistance for ultrasound-guided prostate brachytherapy. Med. Image Anal. 12:535–545, 2008.PubMedCrossRefGoogle Scholar
  16. 16.
    Golemati, S., J. Stoitsis, E. G. Sifakis, T. Balkizas, and K. S. Nikita. Using the Hough transform to segment ultrasound images of longitudinal and transverse sections of the carotid artery. Ultrasound Med. Biol. 33:1918–1932, 2007.PubMedCrossRefGoogle Scholar
  17. 17.
    Gottlieb, R. H., W. B. Robinette, D. J. Rubens, D. F. Hartley, P. J. Fultz, and M.R. Violante. Coating agent permits improved visualization of biopsy needles during sonography. Am. J. Roentgenol. 171:1301–1302, 1998.CrossRefGoogle Scholar
  18. 18.
    Hata, N., J. Tokuda, S. Hurwitz, and S. Morikawa. Mri-compatible manipulator with remotecenter-of-motion control, J. Magn. Reson. Imaging 27:1130–1138, 2008.PubMedCentralPubMedCrossRefGoogle Scholar
  19. 19.
    Hendriks, C., M. van Ginkel, P. Verbeek, and L. J. van Vliet. The generalized radon transform: sampling, accuracy and memory considerations. Pattern Recognit. 38:2494–2505, 2005.CrossRefGoogle Scholar
  20. 20.
    Hong, J., T. Dohi, M. Hashizume, K. Konishi, and N. Hata. An ultrasound-driven needle insertion robot for percutaneous cholecystostomy. Phys. Med. Biol. 49:441, 2000.CrossRefGoogle Scholar
  21. 21.
    Hong, J. S., T. Dohi, M. Hasizume, K. Konishi, and N. Hata. A motion adaptable needle placement instrument based on tumor specific ultrasonic image segmentation. In: Medical Image Computing and Computer-Assisted Intervention MICCAI 2002. Lecture Notes in Computer Science, vol. 2488. Berlin: Springer, pp. 122–129, 2002Google Scholar
  22. 22.
    Hongliang, R., and P. Dupont. Tubular enhanced geodesic active contours for continuum robot detection using 3D ultrasound. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2907–2912, May 2012.Google Scholar
  23. 23.
    Lucas, B., and T. Kanade. An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679, 1981.Google Scholar
  24. 24.
    Mascott, C.R. Comparison of magnetic tracking and optical tracking by simultaneous use of two independent frameless stereotactic systems. Neurosurgery 57:295–301, 2005.PubMedCrossRefGoogle Scholar
  25. 25.
    Mehrabani, B., V. Tavakoli, M. Abolhassani, J. Alirezaie, and A. Ahmadian. An efficient temperature estimation using optical-flow in ultrasound b-mode digital images. In: 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS 2008), pp. 86–89, Aug 2008.Google Scholar
  26. 26.
    Neshat, H., and R. Patel. Real-time parametric curved needle segmentation in 3D ultrasound images. In: 2nd IEEE RAS EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB), pp. 670–675, 2008.Google Scholar
  27. 27.
    Neubach, Z., and M. Shoham. Ultrasound-guided robot for flexible needle steering. IEEE Trans. Biomed.Eng. 57:799–805, 2010.PubMedCrossRefGoogle Scholar
  28. 28.
    Novotny, P., J. A. Stoll, N. V. Vasilyev, P. J. del Nido, P. E. Dupont, T. E. Zickler, and R. D. Howe. GPU based real-time instrument tracking with three-dimensional ultrasound.Med. Image Anal. 11:458–464, 2007.PubMedCentralPubMedCrossRefGoogle Scholar
  29. 29.
    Qiaoliang, L., N. Dong, Y. Wanguan, C. Siping, W. Tianfu, and C. Xin. Use of optical flow to estimate continuous changes in muscle thickness from ultrasound image sequences. Ultrasound Med. Biol. 39:2194–2201, 2013.CrossRefGoogle Scholar
  30. 30.
    Sekhar, S., W. Al-Nuaimy, and A. Nandi. Automated localisation of retinal optic disk using Hough transform. In: 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro(ISBI 2008), pp. 1577–1580, May 2008.Google Scholar
  31. 31.
    Shi, J., and C. Tomasi. Good features to track. In: 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 593–600, 1994.Google Scholar
  32. 32.
    Tanaka, K. A thermomechanical sketch of shape memory effect: one-dimensional tensile behavior. Res. Mech. 18:251–263, 1986.Google Scholar
  33. 33.
    van Stralen, M., K. Leung, M. Voormolen, N. de Jong, A. van der Steen, J. Reiber, and J. Bosch, Time continuous detection of the left ventricular long axis and the mitral valve plane in 3-d echocardiography. Ultrasound Med. Biol. 34:196–207, 2008.PubMedCrossRefGoogle Scholar
  34. 34.
    Vrooijink, G., M. Abayazid, and S.Misra. Real-time three-dimensional flexible needle tracking using two-dimensional ultrasound. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1676–1681, 2013.Google Scholar
  35. 35.
    Yang, B., U. Tan, A. McMillan, R. Gullapalli, and J. Desai. Towards the development of a master-slave surgical system for breast biopsy under continuous MRI. In: 13th International Symposium on Experimental Robotics, Qubec City, Canada, June 2012.Google Scholar

Copyright information

© Biomedical Engineering Society 2014

Authors and Affiliations

  1. 1.Robotics, Automation and Medical Systems (RAMS) Laboratory, Maryland Robotics Center, Institute for Systems ResearchUniversity of MarylandCollege ParkUSA

Personalised recommendations