Improving Needle Detection in 3D Ultrasound Using Orthogonal-Plane Convolutional Networks

  • Arash PourtaherianEmail author
  • Farhad Ghazvinian Zanjani
  • Svitlana Zinger
  • Nenad Mihajlovic
  • Gary Ng
  • Hendrikus Korsten
  • Peter de With
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)


Successful automated detection of short needles during an intervention is necessary to allow the physician identify and correct any misalignment of the needle and the target at early stages, which reduces needle passes and improves health outcomes. In this paper, we present a novel approach to detect needle voxels in 3D ultrasound volume with high precision using convolutional neural networks. Each voxel is classified from locally-extracted raw data of three orthogonal planes centered on it. We propose a bootstrap re-sampling approach to enhance the training in our highly imbalanced data. The proposed method successfully detects 17G and 22G needles with a single trained network, showing a robust generalized approach. Extensive ex-vivo evaluations on 3D ultrasound datasets of chicken breast show 25% increase in F1-score over the state-of-the-art feature-based method. Furthermore, very short needles inserted for only 5 mm in the volume are detected with tip localization errors of \({<}\)0.5 mm, indicating that the tip is always visible in the detected plane.


Needle detection 3D ultrasound Convolutional networks 


  1. 1.
    Barva, M., Uherčík, M., Mari, J.M., Kybic, J., Duhamel, J.R., Liebgott, H., Hlavac, V., Cachard, C.: Parallel integral projection transform for straight electrode localization in 3-D ultrasound images. IEEE Trans. Ultrason. Ferroelect. Freq. Control (UFFC) 55(7), 1559–1569 (2008)CrossRefGoogle Scholar
  2. 2.
    Uherčík, M., Kybic, J., Zhao, Y., Cachard, C., Liebgott, H.: Line filtering for surgical tool localization in 3D ultrasound images. Comput. Biol. Med. 43(12), 2036–2045 (2013)CrossRefGoogle Scholar
  3. 3.
    Beigi, P., Rohling, R., Salcudean, T., Lessoway, V.A., Ng, G.C.: Needle trajectory and tip localization in real-time 3-D ultrasound using a moving stylus. Ultrasound Med. Biol. 41(7), 2057–2070 (2015)CrossRefGoogle Scholar
  4. 4.
    Pourtaherian, A., Mihajlovic, N., Zinger, S., Korsten, H.H.M., de With, P.H.N., Huang, J., Ng, G.C.: Automated in-plane visualization of steep needles from 3D ultrasound volumes. In: Proceedings of IEEE International Ultrasonics Symposium (IUS), pp. 1–4 (2016)Google Scholar
  5. 5.
    Mwikirize, C., Nosher, J.L., Hacihaliloglu, I.: Enhancement of needle tip and shaft from 2D ultrasound using signal transmission maps. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9900, pp. 362–369. Springer, Cham (2016). doi: 10.1007/978-3-319-46720-7_42 CrossRefGoogle Scholar
  6. 6.
    Pourtaherian, A., Zinger, S., de With, P.H.N., Korsten, H.H.M., Mihajlovic, N.: Benchmarking of state-of-the-art needle detection algorithms in 3D ultrasound data volumes. In: Proceedings of SPIE Medical Imaging, vol. 9415, p. 94152B-1-8 (2015)Google Scholar
  7. 7.
    Prasoon, A., Petersen, K., Igel, C., Lauze, F., Dam, E., Nielsen, M.: Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 246–253. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40763-5_31 CrossRefGoogle Scholar
  8. 8.
    Baumgartner, C.F., Kamnitsas, K., Matthew, J., Smith, S., Kainz, B., Rueckert, D.: Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 203–211. Springer, Cham (2016). doi: 10.1007/978-3-319-46723-8_24 CrossRefGoogle Scholar
  9. 9.
    Rowley, H.A., Baluja, S., Kanade, T.: Neural network-based face detection. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 20(1), 23–38 (1998)CrossRefGoogle Scholar
  10. 10.
    Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 15, pp. 315–323 (2011)Google Scholar
  11. 11.
    Tieleman, T., Hinton, G.: Lecture 6.5-RmsProp: divide gradient by running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2), 26–31 (2012)Google Scholar
  12. 12.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)zbMATHMathSciNetGoogle Scholar
  13. 13.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: paradigm for model fitting with applications to image analysis. Commun. ACM 24(6), 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  14. 14.
    van der Maaten, L., Hinton, G.: Visualizing high-dimensional data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Arash Pourtaherian
    • 1
    Email author
  • Farhad Ghazvinian Zanjani
    • 1
  • Svitlana Zinger
    • 1
  • Nenad Mihajlovic
    • 2
  • Gary Ng
    • 3
  • Hendrikus Korsten
    • 4
  • Peter de With
    • 1
  1. 1.Eindhoven University of TechnologyEindhovenThe Netherlands
  2. 2.Philips Research EindhovenEindhovenThe Netherlands
  3. 3.Philips HealthcareBothellUSA
  4. 4.Catharina Hospital EindhovenEindhovenThe Netherlands

Personalised recommendations