Skip to main content
SpringerLink
Log in

Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

We propose an approach of 3D convolutional neural network to segment the prostate in MR images.

Methods

A 3D deep dense multi-path convolutional neural network that follows the framework of the encoder–decoder design is proposed. The encoder is built based upon densely connected layers that learn the high-level feature representation of the prostate. The decoder interprets the features and predicts the whole prostate volume by utilizing a residual layout and grouped convolution. A set of sub-volumes of MR images, centered at the prostate, is generated and fed into the proposed network for training purpose. The performance of the proposed network is compared to previously reported approaches.

Results

Two independent datasets were employed to assess the proposed network. In quantitative evaluations, the proposed network achieved 95.11 and 89.01 Dice coefficients for the two datasets. The segmentation results were robust to variations in MR images. In comparison experiments, the segmentation performance of the proposed network was comparable to the previously reported approaches. In qualitative evaluations, the segmentation results by the proposed network were well matched to the ground truth provided by human experts.

Conclusions

The proposed network is capable of segmenting the prostate in an accurate and robust manner. This approach can be applied to other types of medical images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. de Rooij M, Hamoen EH, Fütterer JJ, Barentsz JO, Rovers MM (2014) Accuracy of multiparametric MRI for prostate cancer detection: a meta-analysis. Am J Roentgenol 202(2):343–351

    Article  Google Scholar 

  2. Fütterer JJ, Briganti A, De Visschere P, Emberton M, Giannarini G, Kirkham A, Taneja SS, Thoeny H, Villeirs G, Villers A (2015) Can clinically significant prostate cancer be detected with multiparametric magnetic resonance imaging? a systematic review of the literature. Eur Urol 68(6):1045–1053

    Article  PubMed  Google Scholar 

  3. Garvey B, Türkbey B, Truong H, Bernardo M, Periaswamy S, Choyke PL (2014) Clinical value of prostate segmentation and volume determination on MRI in benign prostatic hyperplasia. Diagn Interv Radiol 20(3):229

    Article  PubMed  PubMed Central  Google Scholar 

  4. Valerio M, Donaldson I, Emberton M, Ehdaie B, Hadaschik BA, Marks LS, Mozer P, Rastinehad AR, Ahmed HU (2015) Detection of clinically significant prostate cancer using magnetic resonance imaging—ultrasound fusion targeted biopsy: a systematic review. Eur Urol 68(1):8–19

    Article  PubMed  Google Scholar 

  5. Muller BG, Fütterer JJ, Gupta RT, Katz A, Kirkham A, Kurhanewicz J, Moul JW, Pinto PA, Rastinehad AR, Robertson C (2014) The role of magnetic resonance imaging (MRI) in focal therapy for prostate cancer: recommendations from a consensus panel. BJU Int 113(2):218–227

    Article  PubMed  Google Scholar 

  6. Ghai S, Louis AS, Van Vliet M, Lindner U, Haider MA, Hlasny E, Spensieri P, Van Der Kwast TH, McCluskey SA, Kucharczyk W (2015) Real-time MRI-guided focused ultrasound for focal therapy of locally confined low-risk prostate cancer: feasibility and preliminary outcomes. Am J Roentgenol 205(2):W177–W184

    Article  Google Scholar 

  7. Zhu Y, Williams S, Zwiggelaar R (2004) Segmentation of volumetric prostate MRI data using hybrid 2D + 3D shape modeling. In: Proceeding of medical image understanding and analysis, pp 61–64

  8. Allen PD, Graham J, Williamson DC, Hutchinson CE (2006) Differential segmentation of the prostate in MR images using combined 3D shape modelling and voxel classification. In: 3rd IEEE international symposium on biomedical imaging: nano to macro. IEEE, pp 410–413

  9. Freedman D, Radke RJ, Tao Z, Yongwon J, Lovelock DM, Chen GTY (2005) Model-based segmentation of medical imagery by matching distributions. IEEE Trans Med Imaging 24(3):281–292. https://doi.org/10.1109/TMI.2004.841228

    Article  PubMed  Google Scholar 

  10. Makni N, Puech P, Lopes R, Dewalle AS, Colot O, Betrouni N (2008) Combining a deformable model and a probabilistic framework for an automatic 3D segmentation of prostate on MRI. Int J Comput Assist Radiol Surg 4(2):181. https://doi.org/10.1007/s11548-008-0281-y

    Article  PubMed  Google Scholar 

  11. Vikal S, Haker S, Tempany C, Fichtinger G (2009) Prostate contouring in MRI guided biopsy. In: Medical imaging 2009: image processing. International society for optics and photonics, p 72594A

  12. Toth R, Madabhushi A (2012) Multifeature landmark-free active appearance models: application to prostate MRI segmentation. IEEE Trans Med Imaging 31(8):1638–1650

    Article  PubMed  Google Scholar 

  13. Zwiggelaar R, Zhu Y, Williams S (2003) Semi-automatic segmentation of the prostate. In: Perales FJ, Campilho AJC, de la Blanca NP, Sanfeliu A (eds) Pattern recognition and image analysis. Springer, Berlin, pp 1108–1116

    Chapter  Google Scholar 

  14. El Naqa I, Yang D, Apte A, Khullar D, Mutic S, Zheng J, Bradley JD, Grigsby P, Deasy JO (2007) Concurrent multimodality image segmentation by active contours for radiotherapy treatment planning. Med Phys 34(12):4738–4749

    Article  PubMed  Google Scholar 

  15. Klein S, van der Heide UA, Raaymakers BW, Kotte AN, Staring M, Pluim JP (2007) Segmentation of the prostate in MR images by atlas matching. In: 4th IEEE international symposium on biomedical imaging: from nano to macro, 2007. ISBI 2007. IEEE, pp 1300–1303

  16. Flores-Tapia D, Thomas G, Venugopal N, McCurdy B, Pistorius S (2008) Semi automatic MRI prostate segmentation based on wavelet multiscale products. In: Engineering in medicine and biology society, 2008. EMBS 2008. 30th annual international conference of the IEEE. IEEE, pp 3020–3023

  17. Fotin SV, Yin Y, Periaswamy S, Kunz J, Haldankar H, Muradyan N, Cornud F, Turkbey B, Choyke PL (2012) Normalized gradient fields cross-correlation for automated detection of prostate in magnetic resonance images. In: Proceedings of the SPIE, vol 8314. https://doi.org/10.1117/12.911620

  18. Yin Y, Fotin SV, Periaswamy S, Kunz J, Haldankar H, Muradyan N, Cornud F, Turkbey B, Choyke P (2012) Fully automated prostate segmentation in 3D MR based on normalized gradient fields cross-correlation initialization and LOGISMOS refinement. In: Medical imaging 2012: image processing. International Society for Optics and Photonics, p 831406

  19. Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, Shen D (2015) Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108:214–224

    Article  PubMed  PubMed Central  Google Scholar 

  20. Roth HR, Lu L, Farag A, Shin H-C, Liu J, Turkbey EB, Summers RM (2015) Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 556–564

  21. Chen H, Qi X, Yu L, Heng P-A (2016) Dcan: deep contour-aware networks for accurate gland segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2487–2496

  22. Xing F, Xie Y, Yang L (2016) An automatic learning-based framework for robust nucleus segmentation. IEEE Trans Med Imaging 35(2):550–566

    Article  PubMed  Google Scholar 

  23. Cheng R, Roth HR, Lu L, Wang S, Turkbey B, Gandler W, McCreedy ES, Agarwal HK, Choyke P, Summers RM (2016) Active appearance model and deep learning for more accurate prostate segmentation on MRI. In: Medical imaging 2016: image processing. International Society for Optics and Photonics, p 97842I

  24. Zhu Q, Du B, Turkbey B, Choyke PL, Yan P (2017) Deeply-supervised CNN for prostate segmentation. In: 2017 international joint conference on neural networks (IJCNN). IEEE, pp 178–184

  25. Yu L, Yang X, Chen H, Qin J, Heng P-A (2017) Volumetric ConvNets with mixed residual connections for automated prostate segmentation from 3D MR images. In: AAAI, pp 66–72

  26. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  27. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 1520–1528

  28. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  29. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  30. Huang G, Liu Z, Weinberger KQ, van der Maaten L (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 2, p 3

  31. Litjens G, Toth R, van de Ven W, Hoeks C, Kerkstra S, van Ginneken B, Vincent G, Guillard G, Birbeck N, Zhang J, Strand R, Malmberg F, Ou Y, Davatzikos C, Kirschner M, Jung F, Yuan J, Qiu W, Gao Q, Edwards PE, Maan B, van der Heijden F, Ghose S, Mitra J, Dowling J, Barratt D, Huisman H, Madabhushi A (2014) Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge. Med Image Anal 18(2):359–373. https://doi.org/10.1016/j.media.2013.12.002

    Article  PubMed  Google Scholar 

  32. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 5987–5995

  33. Kinga D, Adam JB (2015) A method for stochastic optimization. In: International conference on learning representations (ICLR)

  34. Milletari F, Navab N, Ahmadi S-A (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 4th international conference on 3D vision (3DV). IEEE, pp 565–571

  35. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D U-net: learning dense volumetric segmentation from sparse annotation. In: Medical image computing and computer-assisted intervention—MICCAI 2016. Springer International Publishing, Cham, pp 424–432

    Chapter  Google Scholar 

  36. Tsehay YK, Lay NS, Roth HR, Wang X, Kwak JT, Turkbey BI, Pinto PA, Wood BJ, Summers RM (2017) Convolutional neural network based deep-learning architecture for prostate cancer detection on multiparametric magnetic resonance images. In: SPIE medical imaging. SPIE, p 11

  37. Tae KJ, Sheng X, Wood BJ, Baris T, Choyke PL, Pinto PA, Shijun W, Summers RM (2015) Automated prostate cancer detection using T2-weighted and high-b-value diffusion-weighted magnetic resonance imaging. Med Phys 42(5):2368–2378. https://doi.org/10.1118/1.4918318

    Article  Google Scholar 

Download references

Acknowledgements

This was supported by the National Research Foundation of Korea through the Korea Government (MSIP) under Grant 2016R1C1B2012433.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Sejong University, Seoul, 05006, South Korea

    Minh Nguyen Nhat To, Dang Quoc Vu & Jin Tae Kwak

  2. Molecular Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, 20892, USA

    Baris Turkbey & Peter L. Choyke

Authors
  1. Minh Nguyen Nhat To
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dang Quoc Vu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baris Turkbey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter L. Choyke
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jin Tae Kwak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin Tae Kwak.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

To, M.N.N., Vu, D.Q., Turkbey, B. et al. Deep dense multi-path neural network for prostate segmentation in magnetic resonance imaging. Int J CARS 13, 1687–1696 (2018). https://doi.org/10.1007/s11548-018-1841-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-018-1841-4

Keywords

Search

Navigation