Accurate placement of the needle is critical in interventions like biopsies and regional anesthesia, during which incorrect needle insertion can lead to procedure failure and complications. Therefore, ultrasound guidance is widely used to improve needle placement accuracy. However, at steep and deep insertions, the visibility of the needle is lost. Computational methods for automatic needle tip localization could improve the clinical success rate in these scenarios.
We propose a novel algorithm for needle tip localization during challenging ultrasound-guided insertions when the shaft may be invisible, and the tip has a low intensity. There are two key steps in our approach. First, we enhance the needle tip features in consecutive ultrasound frames using a detection scheme which recognizes subtle intensity variations caused by needle tip movement. We then employ a hybrid deep neural network comprising a convolutional neural network and long short-term memory recurrent units. The input to the network is a consecutive plurality of fused enhanced frames and the corresponding original B-mode frames, and this spatiotemporal information is used to predict the needle tip location.
We evaluate our approach on an ex vivo dataset collected with in-plane and out-of-plane insertion of 17G and 22G needles in bovine, porcine, and chicken tissue, acquired using two different ultrasound systems. We train the model with 5000 frames from 42 video sequences. Evaluation on 600 frames from 30 sequences yields a tip localization error of \(0.52\pm 0.06\) mm and an overall inference time of 0.064 s (15 fps). Comparison against prior art on challenging datasets reveals a 30% improvement in tip localization accuracy.
The proposed method automatically models temporal dynamics associated with needle tip motion and is more accurate than state-of-the-art methods. Therefore, it has the potential for improving needle tip localization in challenging ultrasound-guided interventions.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Marhofer P, Fritsch G (2017) Safe performance of peripheral regional anaesthesia: the significance of ultrasound guidance. Anaesthesia 72(4):431–434
Huang J, Li J, Wang H (2018) The principles and procedures of ultrasound-guided anesthesia techniques. Cureus 10(7):e2980
Kim SY, Chung HW, Oh TS, Lee J (2017) Practical guidelines for ultrasound-guided core needle biopsy of soft-tissue lesions: transformation from beginner to specialist. Korean J Radiol 18(2):361–369
Elsharkawy H, Babazade R, Kolli S, Kalagara H, Soliman ML (2016) The Infiniti plus ultrasound needle guidance system improves needle visualization during the placement of spinal anesthesia. Korean J Anesthesiol 69(4):417–419
Lu H, Li J, Lu Q, Bharat S, Erkamp R, Chen B, Drysdale J, Vignon F, Jain A (2014) A new sensor technology for 2D ultrasound-guided needle tracking. MICCAI 17(Pt. 2):389–396
Xia W, West S, Finlay M, Mari J, Ourselin S, David A, Desjardins A (2017) Looking beyond the imaging plane: 3D needle tracking with a linear array ultrasound probe. Sci Rep 7(1):3674
Arif M, Moelker A, van Walsum T (2018) Needle tip visibility in 3D ultrasound images. Cardiovasc Intervent Radiol 41(1):145–152
Umbarje K, Tang R, Randhawa R, Sawka A, Vaghadia H (2013) Out-of-plane brachial plexus block with a novel SonixGPS (TM) needle tracking system. Anaesthesia 68:433–434
Gadsden J, Latmore M, Levine DM (2015) Evaluation of the eZono 4000 with eZGuide for ultrasound-guided procedures. Expert Rev Med Dev 12:251–261
McVicar J, Niazi AU, Murgatroyd H, Chin KJ, Chan VW (2015) Novice performance of ultrasound-guided needling skills: effect of a needle guidance system. Reg Anesth Pain Med 40:150–153
Najafi M, Abolmaesumi P, Rohling R (2015) Single-camera closed-form real-time needle tracking for ultrasound-guided needle insertion. Ultrasound Med Biol 41:2663–2676
Xia W, Mari JM, West SJ, Ginsberg Y, David AL, Ourselin S, Desjardins AE (2015) In-plane ultrasonic needle tracking using a fiber-optic hydrophone. Med Phys 42:5983–5991
Stolka PJ, Foroughi P, Rendina M, Weiss CR, Hager GD, Boctor EM (2014) Needle guidance using handheld stereo vision and projection for ultrasound-based interventions. MICCAI 17(Pt.2):684–691
Neubash Z, Shoham M (2010) Ultrasound-guided robot for flexible needle steering. IEEE TBME 57(4):799–805
Welleweerd MK, Siepel FJ, Groenhius V, Veltman J, Stramigioli S (2020) Design of an end effector for robot-assisted ultrasound-guided breast biopsies. Int J Comput Assist Radiol Surg 15:681–690
Hatt CR, Ng G, Parthasarathy V (2015) Enhanced needle localization in ultrasound using beam steering and learning-based segmentation. Comput Med Imaging Graph 41:46–54
Beigi P, Rohling R, Salcudean S, Ng G (2017) CASPER: computer-aided segmentation of imperceptible motion—a learning-based tracking of an invisible needle in ultrasound. Int J Comput Assist Radiol Surg 12(11):1857–1866
Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G, Korsten H, With P (2018) Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int J Comput Assist Radiol Surg 13(9):1321–1333
Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Convolutional neural networks for real-time needle detection and localization in 2D ultrasound. Int J Comput Assist Radiol Surg 13(5):647–657
Mwikirize C, Nosher JL, Hacihaliloglu I (2019) Single shot needle localization in 2D ultrasound. In: MICCAI 2019. MICCAI LNCS, 11768, pp 637–645
Mwikirize C, Nosher JL, Hacihaliloglu I (2019) Learning needle tip localization from digital subtraction in 2D ultrasound. Int J Comput Assist Radiol Surg 14(6):1017–1026
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 779–788
This work was funded by Government of Uganda through Makerere University Research and Innovations Fund, and Rutgers TechAdvance Grant.
Conflict of interest
The authors declare that they have no conflict of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
This article does not contain patient data.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Below is the link to the electronic supplementary material.
About this article
Cite this article
Mwikirize, C., Kimbowa, A.B., Imanirakiza, S. et al. Time-aware deep neural networks for needle tip localization in 2D ultrasound. Int J CARS 16, 819–827 (2021). https://doi.org/10.1007/s11548-021-02361-w
- Needle tip localization
- Minimally invasive procedures