Journal of Digital Imaging

, Volume 32, Issue 2, pp 276–282 | Cite as

Predicting Breast Cancer Molecular Subtype with MRI Dataset Utilizing Convolutional Neural Network Algorithm

  • Richard HaEmail author
  • Simukayi Mutasa
  • Jenika Karcich
  • Nishant Gupta
  • Eduardo Pascual Van Sant
  • John Nemer
  • Mary Sun
  • Peter Chang
  • Michael Z. Liu
  • Sachin Jambawalikar


To develop a convolutional neural network (CNN) algorithm that can predict the molecular subtype of a breast cancer based on MRI features. An IRB-approved study was performed in 216 patients with available pre-treatment MRIs and immunohistochemical staining pathology data. First post-contrast MRI images were used for 3D segmentation using 3D slicer. A CNN architecture was designed with 14 layers. Residual connections were used in the earlier layers to allow stabilization of gradients during backpropagation. Inception style layers were utilized deeper in the network to allow learned segregation of more complex feature mappings. Extensive regularization was utilized including dropout, L2, feature map dropout, and transition layers. The class imbalance was addressed by doubling the input of underrepresented classes and utilizing a class sensitive cost function. Parameters were tuned based on a 20% validation group. A class balanced holdout set of 40 patients was utilized as the testing set. Software code was written in Python using the TensorFlow module on a Linux workstation with one NVidia Titan X GPU. Seventy-four luminal A, 106 luminal B, 13 HER2+, and 23 basal breast tumors were evaluated. Testing set accuracy was measured at 70%. The class normalized macro area under receiver operating curve (ROC) was measured at 0.853. Non-normalized micro-aggregated AUC was measured at 0.871, representing improved discriminatory power for the highly represented Luminal A and Luminal B subtypes. Aggregate sensitivity and specificity was measured at 0.603 and 0.958. MRI analysis of breast cancers utilizing a novel CNN can predict the molecular subtype of breast cancers. Larger data sets will likely improve our model.


Breast MRI Molecular subtype CNN 


Compliance with Ethical Standards

Conflict of Interest

The authors declare that there is no conflict of interest.


  1. 1.
    Siegel R, Ma J, Zou Z, Jemal A: Cancer statistics, 2014. CA Cancer J Clin 64:9–29, 2014CrossRefGoogle Scholar
  2. 2.
    Perou CM, Sørlie T, Eisen MB et al.: Molecular portraits of human breast tumours. Nature 406(6797):747–752, 2000CrossRefGoogle Scholar
  3. 3.
    Morris EA: Diagnostic breast MR imaging: current status and future directions. Magn Reson Imaging Clin N Am 18:57–74, 2010CrossRefGoogle Scholar
  4. 4.
    Liberman L, Morris EA, Dershaw DD et al.: MR imaging of the ipsilateral breast in women with percutaneously proven breast cancer. AJR Am J Roentgenol 180(4):901–910, 2003CrossRefGoogle Scholar
  5. 5.
    Schelfout K, Van Goethem M, Kersschot E et al.: Contrast-enhanced MR imaging of breast lesions and effect on treatment. Eur J Surg Oncol. 30(5):501–507, 2004CrossRefGoogle Scholar
  6. 6.
    Sørlie T, Perou CM, Tibshirani R et al.: Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications. Proc Natl Acad Sci U S A 98(19):10869–10874, 2001CrossRefGoogle Scholar
  7. 7.
    Wiechmann L, Sampson M, Stempel M et al.: Presenting features of breast cancer differ by molecular subtype. Ann Surg Oncol 16(10):2705–2710, 2009CrossRefGoogle Scholar
  8. 8.
    Morrow M, Waters J, Morris E: MRI for breast cancer screening, diagnosis, and treatment. Lancet 378:1804–1811, 2011CrossRefGoogle Scholar
  9. 9.
    Goldhirsch A, Wood WC, Coates AS et al.: Strategies for subtypes--dealing with the diversity of breast cancer: highlights of the St. Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer 2011. Ann Oncol 22(8):1736–1747, 2011CrossRefGoogle Scholar
  10. 10.
    Metzger-Filho O, Sun Z, Viale G et al.: Patterns of recurrence and outcome according to breast cancer subtypes in lymph node-negative disease: results from international Breast Cancer Study Group Trials VIII and IX. J Clin Oncol 31(25):3083–3090, 2013CrossRefGoogle Scholar
  11. 11.
    Carey LA, Dees EC, Sawyer L, Gatti L, Moore DT, Collichio F et al.: The triple negative paradox: primary tumor chemosensitivity of breast cancer subtypes. Clin Cancer Res 13(8):2329–2334, 2007CrossRefGoogle Scholar
  12. 12.
    Kumar V, Gu Y, Basu S, Berglund A, Eschrich SA, Schabath MB et al.: Radiomics: the process and the challenges. Magn Reson Imaging 30(9):1234–1248, 2012CrossRefGoogle Scholar
  13. 13.
    Kuo MD, Jamshidi N: Behind the numbers: Decoding molecular phenotypes with radiogenomics—guiding principles and technical considerations. Radiology 270(2):320–325, 2014CrossRefGoogle Scholar
  14. 14.
    Holli-Helenius K, Salminen A, Rinta-Kiikka I et al.: MRI texture analysis in differentiating luminal A and luminal B breast cancer molecular subtypes - a feasibility study. BMC Med Imaging 17(1):69, 2017CrossRefGoogle Scholar
  15. 15.
    Chen W, Giger ML, Lan L, Bick U: Computerized interpretation of breast MRI: investigation of enhancement-variance dynamics. Med Phys 31:1076–1108, 2004CrossRefGoogle Scholar
  16. 16.
    Guo W, Li H, Zhu Y et al.: Prediction of clinical phenotypes in invasive breast carcinomas from the integration of radiomics and genomics data. J Med Imaging (Bellingham) 2:041007, 2015CrossRefGoogle Scholar
  17. 17.
    Fan M, Li H, Wang S et al.: Radiomic analysis reveals DCE-MRI features for prediction of molecular subtypes of breast cancer. PLoS One 12(2):e0171683, 2017CrossRefGoogle Scholar
  18. 18.
    Bhooshan N, Giger ML, Jansen SA et al.: Cancerous breast lesions on dynamic contrast-enhanced MR images: computerized characterization for image-based prognostic markers. Radiology. 254(3):680–690, 2010CrossRefGoogle Scholar
  19. 19.
    Bhooshan N, Giger M, Edwards D et al.: Computerized three-class classification of MRI-based prognostic markers for breast cancer. Phys Med Biol 56(18):5995–6008, 2011CrossRefGoogle Scholar
  20. 20.
    Agner SC, Rosen MA, Englander S et al.: Computerized image analysis for identifying triple-negative breast cancers and differentiating them from other molecular subtypes of breast cancer on dynamic contrast-enhanced MR images: a feasibility study. Radiology 272:91–99, 2014CrossRefGoogle Scholar
  21. 21.
    Mazurowski MA, Zhang J, Grimm LJ et al.: Radiogenomic analysis of breast cancer: luminal B molecular subtype is associated with enhancement dynamics at MR imaging. Radiology 273(2):365–372, 2014CrossRefGoogle Scholar
  22. 22.
    Grimm LJ, Zhang J, Mazurowski MA: Computational approach to radiogenomics of breast cancer: luminal A and luminal B molecular subtypes are associated with imaging features on routine breast MRI extracted using computer vision algorithms. J Magn Reson Imaging 42(4):902–907, 2015CrossRefGoogle Scholar
  23. 23.
    Yamamoto S, Han W, Kim Y et al.: Breast cancer: radiogenomic biomarker reveals associations among dynamic contrast-enhanced MR imaging, long noncoding RNA, and metastasis. Radiology 275(2):384–392, 2015CrossRefGoogle Scholar
  24. 24.
    Ashraf AB, Daye D, Gavenonis S et al.: Identification of intrinsic imaging phenotypes for breast cancer tumors: preliminary associations with gene expression profiles. Radiology 272(2):374–384, 2014CrossRefGoogle Scholar
  25. 25.
    Yamaguchi K, Abe H, Newstread G et al.: Intratumoral heterogeneity of the distribution of kinetic parameters in breast cancer: comparison based on the molecular subtypes of invasive breast cancer. Breast Cancer 22(5):496–502, 2015CrossRefGoogle Scholar
  26. 26.
    Blaschke E, Abe H: MRI phenotype of breast cancer: kinetic assessment for molecular subtypes. J Magn Reson Imaging 42(4):920–924, 2015CrossRefGoogle Scholar
  27. 27.
    LeChun Y, Bengio T, Hinton G: Deep learning. Nature 521:436–444, 2015CrossRefGoogle Scholar
  28. 28.
    Ha R, Jin B, Mango V et al.: Breast cancer molecular subtype as a predictor of the utility of preoperative MRI. AJR Am J Roentgenol 204(6):1354–1360, 2015CrossRefGoogle Scholar
  29. 29.
    Carey LA, Perou CM, Livasy CA et al.: Race, breast cancer subtypes, and survival in the Carolina Breast Cancer Study. JAMA 295:2492–2502, 2006CrossRefGoogle Scholar
  30. 30.
    Nguyen PL, Taghian AG, Katz MS et al.: Breast cancer subtype approximated by estrogen receptor, progesterone receptor, and HER-2 is associated with local and distant recurrence after breast-conserving therapy. J Clin Oncol 26(14):2373–2828, 2008CrossRefGoogle Scholar
  31. 31.
    LeCun Y, Bottou L, Bengio Y et al.: Gradient-based learning applied to document recognition. Proceed IEEE 86(11):2278–2324, 1998CrossRefGoogle Scholar
  32. 32.
    He K, Zhang X, Ren S, et al: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp 770–778Google Scholar
  33. 33.
    Nair, V, Hinton GE: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), 2010, pp 807–814Google Scholar
  34. 34.
    Ioffe S, Szegedy C: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, 2015Google Scholar
  35. 35.
    Srivastava N, Hinton G, Krizhevsky A et al.: Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958, 2014Google Scholar
  36. 36.
    Kingma DP, Jimmy BA: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014Google Scholar
  37. 37.
    Nesterov Y: Gradient methods for minimizing composite objective function. 2007Google Scholar
  38. 38.
    Dozat T: Incorporating nesterov momentum into adam. 2016Google Scholar
  39. 39.
    Glorot X, Bengio Y: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, 2010, pp 249–256Google Scholar
  40. 40.
    Zhu Z, Albadawy E, Saha A, et al: Breast cancer molecular subtype classification using deep features: preliminary results. In: Proceedings Volume 10575, Medical imaging 2018: computer-aided diagnosis; 105752X. 2018Google Scholar
  41. 41.
    Sun C, Shrivastaval A, Singh S, et al: Revisiting unreasonable effectiveness of data in deep learning Era. arXiv preprint arXIV:1707.02968. 2017Google Scholar
  42. 42.
    Guiu S, Michiels S, André F et al.: Molecular subclasses of breast cancer: how do we define them? The IMPAKT 2012 Working Group Statement. Ann Oncol 23(12):2997–3006, 2012CrossRefGoogle Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2019

Authors and Affiliations

  • Richard Ha
    • 1
    Email author
  • Simukayi Mutasa
    • 1
  • Jenika Karcich
    • 1
  • Nishant Gupta
    • 1
  • Eduardo Pascual Van Sant
    • 1
  • John Nemer
    • 1
  • Mary Sun
    • 1
  • Peter Chang
    • 2
  • Michael Z. Liu
    • 3
  • Sachin Jambawalikar
    • 3
  1. 1.Department of RadiologyColumbia University Medical CenterNew YorkUSA
  2. 2.Division of Neuroradiology, Center for Artificial Intelligence in Diagnostic Medicine (CAIDM), Department of Radiological SciencesUCI HealthOrangeUSA
  3. 3.Department of Medical PhysicsColumbia University Medical CenterNew YorkUSA

Personalised recommendations