To present a deep learning–based approach for semi-automatic prostate cancer classification based on multi-parametric magnetic resonance (MR) imaging using a 3D convolutional neural network (CNN).
Two hundred patients with a total of 318 lesions for which histological correlation was available were analyzed. A novel CNN was designed, trained, and validated using different combinations of distinct MRI sequences as input (e.g., T2-weighted, apparent diffusion coefficient (ADC), diffusion-weighted images, and K-trans) and the effect of different sequences on the network’s performance was tested and discussed. The particular choice of modeling approach was justified by testing all relevant data combinations. The model was trained and validated using eightfold cross-validation.
In terms of detection of significant prostate cancer defined by biopsy results as the reference standard, the 3D CNN achieved an area under the curve (AUC) of the receiver operating characteristics ranging from 0.89 (88.6% and 90.0% for sensitivity and specificity respectively) to 0.91 (81.2% and 90.5% for sensitivity and specificity respectively) with an average AUC of 0.897 for the ADC, DWI, and K-trans input combination. The other combinations scored less in terms of overall performance and average AUC, where the difference in performance was significant with a p value of 0.02 when using T2w and K-trans; and 0.00025 when using T2w, ADC, and DWI. Prostate cancer classification performance is thus comparable to that reported for experienced radiologists using the prostate imaging reporting and data system (PI-RADS). Lesion size and largest diameter had no effect on the network’s performance.
The diagnostic performance of the 3D CNN in detecting clinically significant prostate cancer is characterized by a good AUC and sensitivity and high specificity.
• Prostate cancer classification using a deep learning model is feasible and it allows direct processing of MR sequences without prior lesion segmentation.
• Prostate cancer classification performance as measured by AUC is comparable to that of an experienced radiologist.
• Perfusion MR images (K-trans), followed by DWI and ADC, have the highest effect on the overall performance; whereas T2w images show hardly any improvement.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Apparent diffusion coefficient
Area under the curve
Convolutional neural network
Multi-parametric magnetic resonance imaging
Prostate imaging reporting and data system
Receiver operating characteristics
Siegel R, Naishadham D, Jemal A (2013) Cancer statistics. CA Cancer J Clin 63(1):11–30
Johnson LM, Turkbey B, Figg WD, Choyke PL (2014) Multiparametric MRI in prostate cancer management. Nat Rev Clin Oncol 11(6):346–353
Chou R, Croswell JM, Dana T et al (2011) Screening for prostate cancer: a review of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med 155(11):762–771
Schröder FH, Hugosson J, Roobol MJ et al (2009) Screening and prostate-cancer mortality in a randomized European study. N Engl J Med 360(13):1320–1328
Sidhu HS, Benigno S, Ganeshan B et al (2017) Textural analysis of multiparametric MRI detects transition zone prostate cancer. Eur Radiol 27(6):2348–2358
Fehr D, Veeraraghavan H, Wibmer A et al (2015) Automatic classification of prostate cancer Gleason scores from multiparametric magnetic resonance images. Proc Natl Acad Sci U S A 112(46):E6265–E6273
Peng Y, Jiang Y, Yang C et al (2013) Quantitative analysis of multiparametric prostate MR images: differentiation between prostate cancer and normal tissue and correlation with Gleason score--a computer-aided diagnosis development study. Radiology. 267(3):787–796
Valerio M, Donaldson I, Emberton M et al (2015) Detection of clinically significant prostate cancer using magnetic resonance imaging-ultrasound fusion targeted biopsy: a systematic review. Eur Urol 68(1):8–19
Lemaitre G, Martí R, Freixenet J, Vilanova JC, Walker PM, Meriaudeau F (2015) Computer-aided detection and diagnosis for prostate cancer based on mono and multi-parametric MRI: a review. Comput Biol Med 60:8–31
Tiwari P, Kurhanewicz J, Madabhushi A (2013) Multi-kernel graph embedding for detection, Gleason grading of prostate cancer via MRI/MRS. Med Image Anal 17(2):219–235
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. Conference paper at ICLR
Szegedy C, Liu W, Jia Y et al (2015) Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Computer Vision and Pattern Recognition
Huang G, Liu Z, Van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Shin HC, Roth HR, Gao M et al (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298
Tajbakhsh N, Shin JY, Gurudu SR et al (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312
Kasel-Seibert M, Lehmann T, Aschenbach R et al (2016) Assessment of PI-RADS v2 for the detection of prostate cancer. Eur J Radiol 85:726–731
Le MH, Chen J, Wang L et al (2017) Automated diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks. Phys Med Biol 62(16):6497
Yang X, Liu C, Wang Z et al (2017) Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI. Med Image Anal 42:212–227
Mehrtash A, Sedghi A, Ghafoorian A et al (2017) Classification of clinical significance of MRI prostate findings using 3D convolutional neural networks. Proc SPIE Int Soc Opt Eng 10134. pii: 101342A
Fawcett T (2006) An introduction to ROC analysis. Pattern Recogn Lett 27(8):861–874
Litjens G, Debats O, Barentsz J, Karssemeijer N, Huisman H (2017) ProstateX Challenge data. The Cancer Imaging Archive DOI: 10.7937/K9TCIA.2017.MURS5CL
Litjens G, Debats O, Barentsz J, Karssemeijer N, Huisman H (2014) Computer-aided detection of prostate cancer in MRI. IEEE Trans Med Imaging 33:1083–1092
Clark K, Vendt B, Smith K et al (2013) The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6):1045–1057
Kingma DP, Lei Ba J (2017) Adam: a method for stochastic optimization arXiv:1412.6980v9 [cs.LG] 30 Jan.
Lui S, Zheng H, Feng Y, Wi L (2017) Prostate cancer diagnosis using deep learning with 3D multiparametric MRI. In: Medical imaging 2017: computer-aided diagnosis, vol 10134. International Society for Optics and Photonics, p 1013428
This work was funded by the German Research Foundation (Graduate Program BIOQIC, GRK2260, BIOQIC) and the Berlin Institute of Health within the Clinician Scientist Program (TP).
The scientific guarantor of this publication is Prof. Dr. Marc Dewey.
Conflict of interest
The authors of this manuscript declare relationships with the following companies: Nader Aldoj is a PhD student supported by a graduate program of the German Research Foundation (GRK2260, BIOQIC). Prof. Dewey (a PI of BIOQIC funded by the German Research Foundation, GRK2260) has also received grant support from the Heisenberg Program of the DFG for a professorship (DE 1361/14-1) and the FP7 Program of the European Commission for the randomized multi-center DISCHARGE trial (603266-2, HEALTH-2012.2.4.-2). Prof. Dewey is a cardiac section editor of European Radiology. Prof. Dewey has received lecture fees from Toshiba Medical Systems, Guerbet, Cardiac MR Academy Berlin, and Bayer (Schering-Berlex). Tobias Penzkofer received research support from Siemens Healthcare and Philips Healthcare. Tobias Penzkofer received grant support from the Berlin Institute of Health within the Clinician Scientist Program. Outside of the current work, Tobias Penzkofer is involved in clinical trials with AGO, Aprea AB, Astellas Pharma Global Inc., AstraZeneca, Celgene, Genmab A/S, Incyte Corporation, Lion Biotechnologies, Inc., Millennium Pharmaceuticals, Inc., Morphotec Inc., MSD Tesaro Inc., and Roche. Institutional master research agreements exist with Siemens Medical Solutions, Philips Medical Systems, and Toshiba Medical Systems. The terms of these arrangements are managed by the legal department of Charité – Universitätsmedizin Berlin. The other authors declared no conflicts of interest.
Statistics and biometry
No complex statistical methods were necessary for this paper.
No consent was necessary. The dataset is publicly available.
No ethical approval was necessary.
• Performed at one institution
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
About this article
Cite this article
Aldoj, N., Lukas, S., Dewey, M. et al. Semi-automatic classification of prostate cancer on multi-parametric MR imaging using a multi-channel 3D convolutional neural network. Eur Radiol 30, 1243–1253 (2020). https://doi.org/10.1007/s00330-019-06417-z
- Three-dimensional images
- Prostate cancer
- Multi-parametric MRI
- Convolutional neural networks
- Deep learning