Semi-automatic classification of prostate cancer on multi-parametric MR imaging using a multi-channel 3D convolutional neural network



To present a deep learning–based approach for semi-automatic prostate cancer classification based on multi-parametric magnetic resonance (MR) imaging using a 3D convolutional neural network (CNN).


Two hundred patients with a total of 318 lesions for which histological correlation was available were analyzed. A novel CNN was designed, trained, and validated using different combinations of distinct MRI sequences as input (e.g., T2-weighted, apparent diffusion coefficient (ADC), diffusion-weighted images, and K-trans) and the effect of different sequences on the network’s performance was tested and discussed. The particular choice of modeling approach was justified by testing all relevant data combinations. The model was trained and validated using eightfold cross-validation.


In terms of detection of significant prostate cancer defined by biopsy results as the reference standard, the 3D CNN achieved an area under the curve (AUC) of the receiver operating characteristics ranging from 0.89 (88.6% and 90.0% for sensitivity and specificity respectively) to 0.91 (81.2% and 90.5% for sensitivity and specificity respectively) with an average AUC of 0.897 for the ADC, DWI, and K-trans input combination. The other combinations scored less in terms of overall performance and average AUC, where the difference in performance was significant with a p value of 0.02 when using T2w and K-trans; and 0.00025 when using T2w, ADC, and DWI. Prostate cancer classification performance is thus comparable to that reported for experienced radiologists using the prostate imaging reporting and data system (PI-RADS). Lesion size and largest diameter had no effect on the network’s performance.


The diagnostic performance of the 3D CNN in detecting clinically significant prostate cancer is characterized by a good AUC and sensitivity and high specificity.

Key Points

• Prostate cancer classification using a deep learning model is feasible and it allows direct processing of MR sequences without prior lesion segmentation.

• Prostate cancer classification performance as measured by AUC is comparable to that of an experienced radiologist.

• Perfusion MR images (K-trans), followed by DWI and ADC, have the highest effect on the overall performance; whereas T2w images show hardly any improvement.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6



Apparent diffusion coefficient


Area under the curve


Convolutional neural network


Dynamic contrast-enhanced


Diffusion-weighted imaging


Multi-parametric magnetic resonance imaging


Magnetic resonance


Prostate cancer


Prostate imaging reporting and data system


Prostate-specific antigen


Receiver operating characteristics




Transrectal ultrasound


  1. 1.

    Siegel R, Naishadham D, Jemal A (2013) Cancer statistics. CA Cancer J Clin 63(1):11–30

    Article  Google Scholar 

  2. 2.

    Johnson LM, Turkbey B, Figg WD, Choyke PL (2014) Multiparametric MRI in prostate cancer management. Nat Rev Clin Oncol 11(6):346–353

    Article  Google Scholar 

  3. 3.

    Chou R, Croswell JM, Dana T et al (2011) Screening for prostate cancer: a review of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med 155(11):762–771

    Article  Google Scholar 

  4. 4.

    Schröder FH, Hugosson J, Roobol MJ et al (2009) Screening and prostate-cancer mortality in a randomized European study. N Engl J Med 360(13):1320–1328

  5. 5.

    Sidhu HS, Benigno S, Ganeshan B et al (2017) Textural analysis of multiparametric MRI detects transition zone prostate cancer. Eur Radiol 27(6):2348–2358

    Article  Google Scholar 

  6. 6.

    Fehr D, Veeraraghavan H, Wibmer A et al (2015) Automatic classification of prostate cancer Gleason scores from multiparametric magnetic resonance images. Proc Natl Acad Sci U S A 112(46):E6265–E6273

    CAS  Article  Google Scholar 

  7. 7.

    Peng Y, Jiang Y, Yang C et al (2013) Quantitative analysis of multiparametric prostate MR images: differentiation between prostate cancer and normal tissue and correlation with Gleason score--a computer-aided diagnosis development study. Radiology. 267(3):787–796

    Article  Google Scholar 

  8. 8.

    Valerio M, Donaldson I, Emberton M et al (2015) Detection of clinically significant prostate cancer using magnetic resonance imaging-ultrasound fusion targeted biopsy: a systematic review. Eur Urol 68(1):8–19

    Article  Google Scholar 

  9. 9.

    Lemaitre G, Martí R, Freixenet J, Vilanova JC, Walker PM, Meriaudeau F (2015) Computer-aided detection and diagnosis for prostate cancer based on mono and multi-parametric MRI: a review. Comput Biol Med 60:8–31

    Article  Google Scholar 

  10. 10.

    Tiwari P, Kurhanewicz J, Madabhushi A (2013) Multi-kernel graph embedding for detection, Gleason grading of prostate cancer via MRI/MRS. Med Image Anal 17(2):219–235

    Article  Google Scholar 

  11. 11.

    Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. Conference paper at ICLR

  12. 12.

    Szegedy C, Liu W, Jia Y et al (2015) Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA

  13. 13.

    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Computer Vision and Pattern Recognition

  14. 14.

    Huang G, Liu Z, Van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  15. 15.

    Shin HC, Roth HR, Gao M et al (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298

    Article  Google Scholar 

  16. 16.

    Tajbakhsh N, Shin JY, Gurudu SR et al (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312

    Article  Google Scholar 

  17. 17.

    Kasel-Seibert M, Lehmann T, Aschenbach R et al (2016) Assessment of PI-RADS v2 for the detection of prostate cancer. Eur J Radiol 85:726–731

    Article  Google Scholar 

  18. 18.

    Le MH, Chen J, Wang L et al (2017) Automated diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks. Phys Med Biol 62(16):6497

  19. 19.

    Yang X, Liu C, Wang Z et al (2017) Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI. Med Image Anal 42:212–227

    Article  Google Scholar 

  20. 20.

    Mehrtash A, Sedghi A, Ghafoorian A et al (2017) Classification of clinical significance of MRI prostate findings using 3D convolutional neural networks. Proc SPIE Int Soc Opt Eng 10134. pii: 101342A

  21. 21.

    Fawcett T (2006) An introduction to ROC analysis. Pattern Recogn Lett 27(8):861–874

    Article  Google Scholar 

  22. 22.

    Litjens G, Debats O, Barentsz J, Karssemeijer N, Huisman H (2017) ProstateX Challenge data. The Cancer Imaging Archive DOI: 10.7937/K9TCIA.2017.MURS5CL

  23. 23.

    Litjens G, Debats O, Barentsz J, Karssemeijer N, Huisman H (2014) Computer-aided detection of prostate cancer in MRI. IEEE Trans Med Imaging 33:1083–1092

    Article  Google Scholar 

  24. 24.

    Clark K, Vendt B, Smith K et al (2013) The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6):1045–1057

    Article  Google Scholar 

  25. 25.

    Kingma DP, Lei Ba J (2017) Adam: a method for stochastic optimization arXiv:1412.6980v9 [cs.LG] 30 Jan.

  26. 26.

    Lui S, Zheng H, Feng Y, Wi L (2017) Prostate cancer diagnosis using deep learning with 3D multiparametric MRI. In: Medical imaging 2017: computer-aided diagnosis, vol 10134. International Society for Optics and Photonics, p 1013428

Download references


This work was funded by the German Research Foundation (Graduate Program BIOQIC, GRK2260, BIOQIC) and the Berlin Institute of Health within the Clinician Scientist Program (TP).

Author information



Corresponding authors

Correspondence to Nader Aldoj or Marc Dewey.

Ethics declarations


The scientific guarantor of this publication is Prof. Dr. Marc Dewey.

Conflict of interest

The authors of this manuscript declare relationships with the following companies: Nader Aldoj is a PhD student supported by a graduate program of the German Research Foundation (GRK2260, BIOQIC). Prof. Dewey (a PI of BIOQIC funded by the German Research Foundation, GRK2260) has also received grant support from the Heisenberg Program of the DFG for a professorship (DE 1361/14-1) and the FP7 Program of the European Commission for the randomized multi-center DISCHARGE trial (603266-2, HEALTH-2012.2.4.-2). Prof. Dewey is a cardiac section editor of European Radiology. Prof. Dewey has received lecture fees from Toshiba Medical Systems, Guerbet, Cardiac MR Academy Berlin, and Bayer (Schering-Berlex). Tobias Penzkofer received research support from Siemens Healthcare and Philips Healthcare. Tobias Penzkofer received grant support from the Berlin Institute of Health within the Clinician Scientist Program. Outside of the current work, Tobias Penzkofer is involved in clinical trials with AGO, Aprea AB, Astellas Pharma Global Inc., AstraZeneca, Celgene, Genmab A/S, Incyte Corporation, Lion Biotechnologies, Inc., Millennium Pharmaceuticals, Inc., Morphotec Inc., MSD Tesaro Inc., and Roche. Institutional master research agreements exist with Siemens Medical Solutions, Philips Medical Systems, and Toshiba Medical Systems. The terms of these arrangements are managed by the legal department of Charité – Universitätsmedizin Berlin. The other authors declared no conflicts of interest.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

No consent was necessary. The dataset is publicly available.

Ethical approval

No ethical approval was necessary.


• Retrospective

• Performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material


(DOCX 796 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Aldoj, N., Lukas, S., Dewey, M. et al. Semi-automatic classification of prostate cancer on multi-parametric MR imaging using a multi-channel 3D convolutional neural network. Eur Radiol 30, 1243–1253 (2020).

Download citation


  • Three-dimensional images
  • Prostate cancer
  • Multi-parametric MRI
  • Convolutional neural networks
  • Deep learning