Skip to main content

Ultrasound needle segmentation and trajectory prediction using excitation network

International Journal of Computer Assisted Radiology and Surgery volume 15, pages 437–443 (2020)Cite this article

Abstract

Purpose

Ultrasound (US)-guided percutaneous kidney biopsy is a challenge for interventionists as US artefacts prevent accurate viewing of the biopsy needle tip. Automatic needle tracking and trajectory prediction can increase operator confidence in performing biopsies, reduce procedure time, minimize the risk of inadvertent biopsy bleedings, and enable future image-guided robotic procedures.

Methods

In this paper, we propose a tracking-by-segmentation model with spatial and channel “Squeeze and Excitation” (scSE) for US needle detection and trajectory prediction. We adopt a light deep learning architecture (e.g. LinkNet) as our segmentation baseline network and integrate the scSE module to learn spatial information for better prediction. The proposed model is trained with the US images of anonymized kidney biopsy clips from 8 patients. The contour is obtained using the border-following algorithm and area calculated using Green formula. Trajectory prediction is made by extrapolating from the smallest bounding box that can capture the contour.

Results

We train and test our model on a total of 996 images extracted from 102 short videos at a rate of 3 frames per second from each video. A set of 794 images is used for training and 202 images for testing. Our model has achieved IOU of 41.01%, dice accuracy of 56.65%, F1-score of 36.61%, and root-mean-square angle error of 13.3\(^{\circ }\). We are thus able to predict and extrapolate the trajectory of the biopsy needle with decent accuracy for interventionists to better perform biopsies.

Conclusion

Our novel model combining LinkNet and scSE shows a promising result for kidney biopsy application, which implies potential to other similar ultrasound-guided biopsies that require needle tracking and trajectory prediction.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Abolhassani N, Patel R, Moallem M (2007) Needle insertion into soft tissue: a survey. Med Eng Phys 29(4):413–431

    Article  Google Scholar 

  2. Apnorton: green’s theorem and area of polygons. https://math.blogoverflow.com/2014/06/04/greens-theorem-and-area-of-polygons

  3. Beigi P, Rohling R, Salcudean SE, Ng GC (2017) Casper: computer-aided segmentation of imperceptible motion—a learning-based tracking of an invisible needle in ultrasound. Int J Comput Assist Radiol Surg 12(11):1857–1866

    Article  Google Scholar 

  4. Board HP (2018) Singapore renal registry annual report 2016

  5. Bradski G (2000) The OpenCV library. Dr. Dobb’s J Softw Tools 25:120–125

    Google Scholar 

  6. Chaurasia A, Culurciello E (2017) Linknet: exploiting encoder representations for efficient semantic segmentation. CoRR http://arxiv.org/abs/1707.03718

  7. Chen LC, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587

  8. Fitzgibbon AW, Fisher RB (1996) A buyer’s guide to conic fitting. DAI Research paper

  9. Foley JD, Van FD, Van Dam A, Feiner SK, Hughes JF, Hughes J, Angel E (1996) Computer graphics: principles and practice, vol 12110. Addison-Wesley Professional, Boston

    Google Scholar 

  10. Hamper UM, Savader BL, Sheth S (1991) Improved needle-tip visualization by color doppler sonography. AJR Am J Roentgenol 156(2):401–402

    CAS  Article  Google Scholar 

  11. Hatt CR, Ng G, Parthasarathy V (2015) Enhanced needle localization in ultrasound using beam steering and learning-based segmentation. Comput Med Imaging Graph 41:46–54

    Article  Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. CoRR http://arxiv.org/abs/1512.03385

  13. Johnson RJ, Feehally J, Floege J (2014) Comprehensive clinical nephrology E-Book. Elsevier Health Sciences, Amsterdam

    Google Scholar 

  14. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  15. Mathiassen K, Dall’Alba D, Muradore R, Fiorini P, Elle OJ (2016) Robust real-time needle tracking in 2-D ultrasound images using statistical filtering. IEEE Trans Control Syst Technol 25(3):966–978

    Article  Google Scholar 

  16. Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Convolution neural networks for real-time needle detection and localization in 2D ultrasound. Int J Comput Assist Radiol Surg 13(5):647–657

    Article  Google Scholar 

  17. Neubach Z, Shoham M (2010) Ultrasound-guided robot for flexible needle steering. IEEE Trans Biomed Eng 57(4):799–805

    Article  Google Scholar 

  18. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch. In: NIPS 2017 Autodiff workshop: the future of gradient-based machine learning software and techniques. Long Beach, CA, US

  19. Peng C, Zhang X, Yu G, Luo G, Sun J (2017) Large kernel matters—improve semantic segmentation by global convolutional network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4353–4361

  20. Roy AG, Navab N, Wachinger C (2018) Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks. In: International conference on medical image computing and computer-assisted intervention, pp 421–429. Springer

  21. Suzuki S, Keiichi B (1985) Topological structural analysis of digitized binary images by border following. Comput Vis Graph Image Process 30(1):32–46

    Article  Google Scholar 

  22. Teh CH, Chin RT (1989) On the detection of dominant points on digital curves. IEEE Trans Pattern Anal Mach Intell 11(8):859–872

    Article  Google Scholar 

  23. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Iccv, vol 98, p 2

  24. Vrooijink GJ, Abayazid M, Misra S (2013) Real-time three-dimensional flexible needle tracking using two-dimensional ultrasound. In: 2013 IEEE international conference on robotics and automation (ICRA), pp 1688–1693. IEEE (2013)

  25. Zhao Y, Cachard C, Liebgott H (2013) Automatic needle detection and tracking in 3D ultrasound using an ROI-based RANSAC and Kalman method. Ultrason Imaging 35(4):283–306

    Article  Google Scholar 

  26. Zhao Z, Xu S, Wood B, Tse ZTH (2018) An electromagnetic tracking needle clip: an enabling design for low-cost image-guided therapy. In: 2018 Design of medical devices conference. American Society of Mechanical Engineers, , pp V001T07A010–V001T07A010

Download references

Funding

This study was funded by NMRC Bedside & Bench under Grant R-397-000-245-511 awarded to Dr. Hongliang Ren.

Author information

Authors and Affiliations

  1. Faculty of Engineering, National University of Singapore, Singapore, Singapore

    Jia Yi Lee, Mobarakol Islam, Jing Ru Woh, T. S. Mohamed Washeem & Hongliang Ren

  2. Department of Biomedical Engineering, National University of Singapore, Singapore, Singapore

    Mobarakol Islam, T. S. Mohamed Washeem & Hongliang Ren

  3. NUS Graduate School for Integrative Sciences and Engineering (NGS), NUS, Singapore, Singapore

    Mobarakol Islam

  4. Division of Nephrology, National University Hospital, Singapore, Singapore

    Lee Ying Clara Ngoh & Weng Kin Wong

  5. Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore

    Lee Ying Clara Ngoh

Authors
  1. Jia Yi Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mobarakol Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Ru Woh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. T. S. Mohamed Washeem
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lee Ying Clara Ngoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Weng Kin Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hongliang Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongliang Ren.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study. If articles do not contain patient data, the statement should read: This article does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lee, J.Y., Islam, M., Woh, J.R. et al. Ultrasound needle segmentation and trajectory prediction using excitation network. Int J CARS 15, 437–443 (2020). https://doi.org/10.1007/s11548-019-02113-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-019-02113-x

Keywords

  • Ultrasound imaging
  • Needle tracking
  • Minimally invasive surgery
  • Concurrent Spatial and Channel “Squeeze and Excitation”
  • LinkNet