Abstract
Purpose
Robot-assisted needle insertion guided by 2D ultrasound (US) can effectively improve the accuracy and success rate of clinical puncture. To this end, automatic and accurate needle-tracking methods are important for monitoring puncture processes, avoiding the needle deviating from the intended path, and reducing the risk of injury to surrounding tissues. This work aims to develop a framework for automatic and accurate detection of an inserted needle in 2D US image during the insertion process.
Methods
We propose a novel convolutional neural network architecture comprising of a two-channel encoder and single-channel decoder for needle segmentation using needle motion information extracted from two adjacent US image frames. Based on the novel network, we further propose an automatic needle detection framework. According to the prediction result of the previous frame, a region of interest of the needle in the US image was extracted and fed into the proposed network to achieve finer and faster continuous needle localization.
Results
The performance of our method was evaluated based on 1000 pairs of US images extracted from robot-assisted needle insertions on freshly excised bovine and porcine tissues. The needle segmentation network achieved 99.7% accuracy, 86.2% precision, 89.1% recall, and an F1-score of 0.87. The needle detection framework successfully localized the needle with a mean tip error of 0.45 ± 0.33 mm and a mean orientation error of 0.42° ± 0.34° and achieved a total processing time of 50 ms per image.
Conclusion
The proposed framework demonstrated the capability to realize robust, accurate, and real-time needle localization during robot-assisted needle insertion processes. It has a promising application in tracking the needle and ensuring the safety of robotic-assisted automatic puncture during challenging US-guided minimally invasive procedures.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Soni NJ, Franco-Sadud R, Schnobrich D, Dancel R, Tierney DM, Salame G, Restrepo MI, McHardy P (2016) Ultrasound guidance for lumbar puncture. Neurol Clin Pract 6(4):358–368
Chen SH, Wang F, Lin YP, Shi QS, Wang YL (2021) Ultrasound-guided needle insertion robotic system for percutaneous puncture. Int J Comput Assist Radiol Surg 16(3):475–484
Morse J, Terrasini N, Wehbe M, Philippona C, Zaouter C, Cyr S, Hemmerling TM (2014) Comparison of success rates, learning curves, and inter-subject performance variability of robot-assisted and manual ultrasound-guided nerve block needle guidance in simulation. Br J Anaesth 112(6):1092–1097
Zhao Y, Shen Y, Bernard A, Cachard C, Liebgott H (2017) Evaluation and comparison of current biopsy needle localization and tracking methods using 3D ultrasound. Ultrasonics 73:206–220
Zhao Y, Liebgott H, Cachard C (2015) Comparison of the existing tool localisation methods on two-dimensional ultrasound images and their tracking results. IET Contr Theory Appl 9(7):1180–1180
Zhao Z, Tse ZTH (2019) An electromagnetic tracking needle clip: an enabling design for low-cost image-guided therapy. Minim Invasive Ther Allied Technol 28(3):165–171
Xia WF, West SJ, Finlay MC, Mari JM, Ourselin S, David AL, Desjardins AE (2017) Looking beyond the imaging plane: 3D needle tracking with a linear array ultrasound probe. Sci Rep 7:3674. https://doi.org/10.1038/s41598-017-03886-4
Ayvali E, Desai JP (2015) Optical flow-based tracking of needles and needle-tip localization using circular hough transform in ultrasound images. Ann Biomed Eng 43(8):1828–1840
Barva M, Uhercik M, Mari JM, Kybic J, Duhamel JR, Liebgott H, Hlavac V, Cachard C (2008) Parallel integral projection transform for straight electrode localization in 3-D ultrasound images. IEEE Trans Ultrason Ferroelectr Freq Control 55(7):1559–1569
Mathiassen K, Dall’Alba D, Muradore R, Fiorini P, Elle OJ (2017) Robust real-time needle tracking in 2-D ultrasound images using statistical filtering. IEEE Trans Control Syst Technol 25(3):966–978
Kaya M, Bebek O (2014) Needle localization using Gabor filtering in 2D ultrasound images. In: IEEE international conference on robotics and automation (ICRA), pp 4881–4886
Zhao Y, Cachard C, Liebgott H (2013) Automatic needle detection and tracking in 3D ultrasound using an ROI-based RANSAC and Kalman method. Ultrason Imaging 35(4):283–306
Beigi P, Rohling R, Salcudean SE, Ng GC (2016) Spectral analysis of the tremor motion for needle detection in curvilinear ultrasound via spatiotemporal linear sampling. Int J Comput Assist Radiol Surg 11(6):1183–1192
Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sanchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88
Chen AI, Balter ML, Maguire TJ, Yarmush ML (2020) Deep learning robotic guidance for autonomous vascular access. Nat Mach Intell 2(2):104–115
Park I, Kim HK, Chung WK, Kim K (2020) Deep learning based real-time OCT image segmentation and correction for robotic needle insertion systems. IEEE Robot Autom Let 5(3):4517–4524
Maris B, Tenga C, Vicario R, Palladino L, Murr N, De Piccoli M, Calanca A, Puliatti S, Micali S, Tafuri A, Fiorini P (2021) Toward autonomous robotic prostate biopsy: a pilot study. Int J Comput Assist Radiol Surg 16(8):1393–1401
Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Convolution neural networks for real-time needle detection and localization in 2D ultrasound. Int J Comput Assist Radiol Surg 13(5):647–657
Lee JY, Islam M, Woh JR, Washeem TSM, Ngoh LYC, Wong WK, Ren H (2020) Ultrasound needle segmentation and trajectory prediction using excitation network. Int J Comput Assist Radiol Surg 15(3):437–443
Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng GC, Korsten HHM, de With PHN (2018) Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int J Comput Assist Radiol Surg 13(9):1321–1333
Arif M, Moelker A, van Walsum T (2019) Automatic needle detection and real-time Bi-planar needle visualization during 3D ultrasound scanning of the liver. Med Image Anal 53:104–110
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention (MICCAI), pp 234–241
Milletari F, Navab N, Ahmadi SA (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings of 2016 fourth international conference on 3d vision (3DV), pp 565–571
Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the 3rd international conference in learning representations (ICLR)
Acknowledgements
We would like to thank Editage (www.editage.cn) for English language editing.
Funding
This study was funded by the National Natural Science Foundation of China (Grant number 52175020).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Human and animals rights
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. No animal experiments were performed in this study.
Informed consent
Informed consent was obtained from all participants.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, S., Lin, Y., Li, Z. et al. Automatic and accurate needle detection in 2D ultrasound during robot-assisted needle insertion process. Int J CARS 17, 295–303 (2022). https://doi.org/10.1007/s11548-021-02519-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-021-02519-6