Skip to main content
SpringerLink
Log in

A Deep Learning-Based Approach for the Detection and Localization of Prostate Cancer in T2 Magnetic Resonance Images

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

We address the problem of prostate lesion detection, localization, and segmentation in T2W magnetic resonance (MR) images. We train a deep convolutional encoder-decoder architecture to simultaneously segment the prostate, its anatomical structure, and the malignant lesions. To incorporate the 3D contextual spatial information provided by the MRI series, we propose a novel 3D sliding window approach, which preserves the 2D domain complexity while exploiting 3D information. Experiments on data from 19 patients provided for the public by the Initiative for Collaborative Computer Vision Benchmarking (I2CVB) show that our approach outperforms traditional pattern recognition and machine learning approaches by a significant margin. Particularly, for the task of cancer detection and localization, the system achieves an average AUC of 0.995, an accuracy of 0.894, and a recall of 0.928. The proposed mono-modal deep learning-based system performs comparably to other multi-modal MR-based systems. It could improve the performance of a radiologist in prostate cancer diagnosis and treatment planning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Alkadi R, Taher F, El-Baz A, Naoufel W: Early diagnosis and staging of prostate cancer using magnetic resonance imaging: State of the art and perspectives. In: Prostate cancer imaging: An engineering and clinical perspective, chapter 2. Taylor & Francis, In-press

  2. Badrinarayanan V, Kendall A, Cipolla R: Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39 (12): 2481–2495, 2017

    Article  PubMed  Google Scholar 

  3. Chan I, Wells W, Mulkern RV, Haker S, Zhang J, Zou KH, Maier SE, Tempany C: Detection of prostate cancer by integration of line-scan diffusion, t2-mapping and t2-weighted magnetic resonance imaging; a multichannel statistical classifier. Med Phys 30 (9): 2390–2398, 2003

    Article  PubMed  Google Scholar 

  4. Chawla NV, Bowyer KW, Hall LO, Philip Kegelmeyer W: Smote: synthetic minority over-sampling technique. J Artif Intell Res 16: 321–357, 2002

    Article  Google Scholar 

  5. Csurka G, Larlus D, Perronnin F, Meylan F: What is a good evaluation measure for semantic segmentation?. In: BMVC, volume 27, p 2013. Citeseer, 2013

  6. Drozdzal M, Chartrand G, Vorontsov E, Shakeri M, Di Jorio L, An T, Romero A, Bengio Y, Pal C, Kadoury S: Learning normalized inputs for iterative estimation in medical image segmentation. Med Image Anal 44: 1–13, 2018

    Article  PubMed  Google Scholar 

  7. Eigen D, Fergus R: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture.. In: Proceedings of the IEEE international conference on computer vision, 2015, pp 2650–2658

  8. Ferlay J, Soerjomataram I, Dikshit R, Eser S, Mathers C, Rebelo M, Parkin DM, Forman D, Bray F: Cancer incidence and mortality worldwide: sources, methods and major patterns in globocan 2012. Int J Cancer 136 (5): E359–86, 2015

    Article  CAS  PubMed  Google Scholar 

  9. Fütterer JJ: Multiparametric mri in the detection of clinically significant prostate cancer. Korean J Radiol 18 (4): 597–606, 2017

    Article  PubMed  PubMed Central  Google Scholar 

  10. Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Garcia-Rodriguez J: A review on deep learning techniques applied to semantic segmentation, 2017. arXiv:1704.06857

  11. Greenspan H, van Ginneken B, Summers RM: Guest editorial deep learning in medical imaging overview and future promise of an exciting new technique. IEEE Trans Med Imaging 35 (5): 1153–1159, 2016

    Article  Google Scholar 

  12. Guo Y, Gao Y, Shen D: Deformable mr prostate segmentation via deep feature learning and sparse patch matching. IEEE Trans Med Imaging 35 (4): 1077–1089, 2016

    Article  PubMed  Google Scholar 

  13. Hall MA: Correlation-based feature selection of discrete and numeric class machine learning, 2000

  14. Han H, Wang W-Y, Mao B-H: Borderline-smote: a new over- sampling method in imbalanced data sets learning.. In: International Conference on Intelligent Computing, pp 878–887. Springer , 2005

  15. He K, Zhang X, Ren S, Sun J: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034, 2015

  16. Kiraly AP, Nader CA, Tuysuzoglu A, Grimm R, Kiefer B, El-Zehiry N, Kamen A: Deep convolutional encoder-decoders for prostate cancer detection and classification.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 489–497. Springer, 2017

  17. Kohl S, Bonekamp D, Schlemmer H-P, Yaqubi K, Hohenfellner M, Hadaschik B, Radtke J-P, Maier-Hein K: Adversarial networks for the detection of aggressive prostate cancer, 2017. arXiv:1702.08014 1702.08014

  18. Kumar D, Wong A, Clausi DA: Lung nodule classification using deep features in ct images.. In: 2015 12th conference on computer and robot vision (CRV), pp 133–138. IEEE, 2015

  19. Lemaitre G: Computer-aided diagnosis for prostate cancer using multi-parametric magnetic resonance imaging. PhD thesis, Ph. D. dissertation, Universitat de Girona and Université de Bourgogne, 2016

  20. Lemaître G, Martí R, Freixenet J, Vilanova JC, Walker PM, Meriaudeau F: Computer-aided detection and diagnosis for prostate cancer based on mono and multi-parametric mri: A review. Comput Biol Med 60: 8–31, 2015

    Article  PubMed  Google Scholar 

  21. Lemaitre G, Martí R, Rastgoo M, Mériaudeau F: Computer-aided detection for prostate cancer detection based on multi-parametric magnetic resonance imaging.. In: Engineering in Medicine and Biology Society (EMBC), 2017 39th Annual International Conference of the IEEE, pp 3138–3141. IEEE, 2017

  22. Litjens G, Debats O, Barentsz J, Karssemeijer N, Huisman H: Computer-aided detection of prostate cancer in mri. IEEE Trans Med Imaging 33 (5): 1083–1092, 2014

    Article  PubMed  Google Scholar 

  23. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI: A survey on deep learning in medical image analysis. Med Image Anal 42: 60–88, 2017

    Article  PubMed  Google Scholar 

  24. Long J, Shelhamer E, Darrell T: Fully convolutional networks for semantic segmentation.. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp 3431–3440

  25. Lv D, Guo X, Wang X, Zhang J, Fang J: Computerized characterization of prostate cancer by fractal analysis in mr images. J Magn Reson Imaging 30 (1): 161–168, 2009

    Article  PubMed  Google Scholar 

  26. Mani I, Zhang I: knn approach to unbalanced data distributions: a case study involving information extraction.. In: Proceedings of workshop on learning from imbalanced datasets, vol 126, 2003

  27. Mazurowski MA, Buda M, Saha A, Bashir MR: Deep learning in radiology: an overview of the concepts and a survey of the state of the art, 2018. arXiv:1802.08717

  28. Puech P, Betrouni N, Makni N, Dewalle A-S, Villers A, Lemaitre L: Computer-assisted diagnosis of prostate cancer using dce-mri data: design, implementation and preliminary results. Int J Comput. Assist Radiol Surg 4 (1): 1–10, 2009

    Article  PubMed  Google Scholar 

  29. Qi CR, Hao SU, Nießner M, Dai A, Yan M, Guibas LJ: Volumetric and multi-view cnns for object classification on 3d data.. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp 5648–5656

  30. Rampun A, Zheng L, Malcolm P, Tiddeman B, Zwiggelaar R: Computer-aided detection of prostate cancer in t2-weighted mri within the peripheral zone. Phys Med Biol 61 (13): 4796, 2016

    Article  CAS  PubMed  Google Scholar 

  31. Reda I, Shalaby A, Khalifa F, Elmogy M, Aboulfotouh A, El-Ghar MA, Hosseini-Asl E, Werghi N, Keynton R, El-Baz A: Computer-aided diagnostic tool for early detection of prostate cancer.. In: IEEE international conference on image processing (ICIP), pp 2668–2672. IEEE, 2016

  32. Ronneberger O, Fischer P, Brox T: U-net: Convolutional networks for biomedical image segmentation.. In: International Conference on Medical image computing and computer-assisted intervention, pp 234–241. Springer, 2015

  33. Roth HR, Lu Le, Liu J, Yao J, Seff A, Cherry K, Kim L, Summers RM: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 35 (5): 1170–1181, 2016

    Article  PubMed  Google Scholar 

  34. Roth HR, Lu L, Seff A, Cherry KM, Hoffman J, Wang S, Liu J, Turkbey E, Summers RM: A new 2.5 d representation for lymph node detection using random sets of deep convolutional neural network observations.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 520–527. Springer, 2014

  35. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L: ImageNet large scale visual recognition challenge. Int J Comput Vis 115 (3): 211–252, 2015

    Article  Google Scholar 

  36. Simonyan K, Zisserman A: Very deep convolutional networks for large-scale image recognition, 2014. arXiv:1409.1556

  37. Smith MR, Martinez T, Giraud-Carrier C: An instance level analysis of data complexity. Mach Learn 95 (2): 225–256, 2014

    Article  Google Scholar 

  38. Tiwari P, Kurhanewicz J, Madabhushi A: Multi-kernel graph embedding for detection, gleason grading of prostate cancer via mri/mrs. Med Image Anal 17 (2): 219–235, 2013

    Article  PubMed  Google Scholar 

  39. Tiwari P, Rosen M, Madabhushi A: A hierarchical spectral clustering and nonlinear dimensionality reduction scheme for detection of prostate cancer from magnetic resonance spectroscopy (mrs). Med Phys 36 (9Part1): 3927–3939, 2009

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Tiwari P, Viswanath S, Kurhanewicz J, Sridhar A, Madabhushi A: Multimodal wavelet embedding representation for data combination (maweric): integrating magnetic resonance imaging and spectroscopy for prostate cancer detection. NMR Biomed 25 (4): 607–619, 2012

    Article  CAS  PubMed  Google Scholar 

  41. Trigui R, Mitéran J, Walker PM, Sellami L, Ben Hamida A: Automatic classification and localization of prostate cancer using multi-parametric mri/mrs. Biomed Signal Process Control 31: 189–198, 2017

    Article  Google Scholar 

  42. Tustison NJ, Avants BB, Cook PA, Zheng Y, Egan A, Yushkevich PA, Gee JC: N4itk: Improved n3 bias correction. IEEE Trans Med Imaging 29 (6): 1310–1320, 2010

    Article  PubMed  PubMed Central  Google Scholar 

  43. Viswanath S, Bloch BN, Genega E, Rofsky N, Lenkinski R, Chappelow J, Toth R, Madabhushi A: A comprehensive segmentation, registration, and cancer detection scheme on 3 tesla in vivo prostate dce-mri.. In: International conference on medical image computing and computer-assisted intervention, pp 662–669. Springer, 2008

  44. Viswanath SE, Bloch NB, Chappelow JC, Toth R, Rofsky NM, Genega EM, Lenkinski RE, Madabhushi A: Central gland and peripheral zone prostate tumors have significantly different quantitative imaging signatures on 3 tesla endorectal, in vivo t2- weighted mr imagery. J Magn Reson Imaging 36 (1): 213–224, 2012

    Article  PubMed  PubMed Central  Google Scholar 

  45. Vos PC, Barentsz JO, Karssemeijer N, Huisman HJ: Automatic computer-aided detection of prostate cancer based on multiparametric magnetic resonance image analysis. Phys Med Biol 57 (6): 1527, 2012

    Article  CAS  PubMed  Google Scholar 

  46. Wang L, Zwiggelaar R: 3d texton based prostate cancer detection using multiparametric magnetic resonance imaging.. In: Annual conference on medical image understanding and analysis, pp 309– 319. Springer, 2017

  47. Wang Z, Liu C, Cheng D, Wanga L, Yang X, Chengb K-TT: Automated detection of clinically significant prostate cancer in mp-mri images based on an end-to-end deep neural network. IEEE Transactions on Medical Imaging, 2018

  48. Zhirong WU, Song S, Khosla A, Fisher YU, Zhang L, Tang X, Xiao J: 3d shapenets; A deep representation for volumetric shapes.. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp 1912–1920

  49. Yang X, Liu C, Wang Z, Yang J, Min HL, Wang L, Cheng K-TT: Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric mri. Med Image Anal 42: 212–227, 2017

    Article  PubMed  Google Scholar 

  50. Yang X, Wang Z, Liu C, Le HM, Chen J, Cheng K-TT, Wang L: Joint detection and diagnosis of prostate cancer in multi-parametric mri based on multimodal convolutional neural networks.. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 426–434. Springer, 2017

  51. Lequan YU, Yang X, Chen H, Qin J, Heng P-A: Volumetric convnets with mixed residual connections for automated prostate segmentation from 3d mr images.. In: AAAI, 2017, pp 66–72

  52. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A: Learning deep features for discriminative localization.. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2921–2929. IEEE, 2016

Download references

Acknowledgements

The authors would also like to thank Dr. Waleed Hassen and Dr. Eric Vens from Cleveland Clinic, Abu Dhabi, and Dr. Salah El-Rai from Sheikh Khalifa General Hospital for their support and collaboration.

Funding

This work is support by a research grant from Al-Jalila foundation ref. AJF-201616.

Author information

Authors and Affiliations

  1. Khalifa University of Science and Technology, PO Box 127788, Abu Dhabi, United Arab Emirates

    Ruba Alkadi, Fatma Taher & Naoufel Werghi

  2. University of Louisville, Louisville, KY, 40292, USA

    Ayman El-baz

Authors
  1. Ruba Alkadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fatma Taher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ayman El-baz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naoufel Werghi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruba Alkadi.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

http://i2cvb.github.io/

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alkadi, R., Taher, F., El-baz, A. et al. A Deep Learning-Based Approach for the Detection and Localization of Prostate Cancer in T2 Magnetic Resonance Images. J Digit Imaging 32, 793–807 (2019). https://doi.org/10.1007/s10278-018-0160-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-018-0160-1

Keywords

Search

Navigation