A Multi-scale CNN and Curriculum Learning Strategy for Mammogram Classification

  • William LotterEmail author
  • Greg Sorensen
  • David Cox
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10553)


Screening mammography is an important front-line tool for the early detection of breast cancer, and some 39 million exams are conducted each year in the United States alone. Here, we describe a multi-scale convolutional neural network (CNN) trained with a curriculum learning strategy that achieves high levels of accuracy in classifying mammograms. Specifically, we first train CNN-based patch classifiers on segmentation masks of lesions in mammograms, and then use the learned features to initialize a scanning-based model that renders a decision on the whole image, trained end-to-end on outcome data. We demonstrate that our approach effectively handles the “needle in a haystack” nature of full-image mammogram classification, achieving 0.92 AUROC on the DDSM dataset.


  1. 1.
    Arevalo, J., Gonzalez, F., Ramos-Pollan, R., et al.: Representation learning for mammography mass lesion classification with convolutional neural networks. Comput. Methods Programs Biomed. 127, 248–257 (2016)CrossRefGoogle Scholar
  2. 2.
    Bengio, Y., Louradour, J., Collobert, R., et al.: Curriculum learning. In: ICML (2009)Google Scholar
  3. 3.
    Berry, D.A., Cronin, K.A., Plevritis, S.K., et al.: Effect of screening and adjuvant therapy on mortality from breast cancer. In: NEJM (2005)Google Scholar
  4. 4.
    Brewer, N.T., Salz, T., Lillie, S.E.: Systematic review: the long-term effects of false-positive mammograms. Ann. Internal Med. 146(7), 502–510 (2007)CrossRefGoogle Scholar
  5. 5.
    Cancer Stat Facts: Female breast cancer.
  6. 6.
    Carneiro, G., Nascimento, J., Bradley, A.P.: Unregistered multiview mammogram analysis with pre-trained deep learning models. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 652–660. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_78 CrossRefGoogle Scholar
  7. 7.
    Dhungel, N., Carneiro, G., Bradley, A.P.: Deep structured learning for mass segmentation from mammograms. arXiv:1410.7454 (2014)
  8. 8.
    Dhungel, N., Carneiro, G., Bradley, A.P.: Automated mass detection in mammograms using cascaded deep learning and random forests. In: DICTA (2015)Google Scholar
  9. 9.
    Elmore, J.G., Jackson, S.L., Abraham, L., et al.: Variability in interpretive performance at screening mammography and radiologists characteristics associated with accuracy. Radiology 253(3), 587–589 (2009)CrossRefGoogle Scholar
  10. 10.
    Geras, K.J., Wolfson, S., Kim, S.G., et al.: High-resolution breast cancer screening with multi-view deep convolutional neural networks. arXiv:1703.07047 (2017)
  11. 11.
    He, K., Zhang, X., Ren, X., Sun, J.: Deep residual learning for image recognition. arXiv:1512.03385 (2012)
  12. 12.
    Heath, M., Bowyer, K., Kopans, D., et al.: The digital database for screening mammography. In: Proceedings of the Fifth International Workshop on Digital Mammography (2001)Google Scholar
  13. 13.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167 (2015)
  14. 14.
    Kooi, T., Litjens, G., van Ginneken, B., et al.: Large scale deep learning for computer aided detection of mammographic lesions. Med. Image Anal. 35, 303–312 (2017)CrossRefGoogle Scholar
  15. 15.
    Lehman, C., Arao, R., Sprague, B., et al.: National performance benchmarks for modern screening digital mammography: update from the breast cancer surveillance consortium. Radiology 283(1), 49–58 (2017)CrossRefGoogle Scholar
  16. 16.
    Levy, D., Jain, A.: Breast mass classification from mammograms using deep convolutional neural networks. arXiv:1612.00542 (2016)
  17. 17.
    Mordang, J.-J., Janssen, T., Bria, A., Kooi, T., Gubern-Mérida, A., Karssemeijer, N.: Automatic microcalcification detection in multi-vendor mammography using convolutional neural networks. In: Tingberg, A., Lång, K., Timberg, P. (eds.) IWDM 2016. LNCS, vol. 9699, pp. 35–42. Springer, Cham (2016). doi: 10.1007/978-3-319-41546-8_5 Google Scholar
  18. 18.
    Moreira, I.C., Amaral, I., Domingues, I., et al.: INbreast: toward a full-field digital mammographic database. Acad. Radiol. 19(2), 236–248 (2012)CrossRefGoogle Scholar
  19. 19.
    Myers, E.R., Moorman, P., Gierisch, J.M., et al.: Benefits and harms of breast cancer screening: a systematic review. JAMA 314(15), 1615–1634 (2015)CrossRefGoogle Scholar
  20. 20.
    Nishikawa, R.M.: Current status and future directions of computer-aided diagnosis in mammography. Comput. Med. Imaging Graph. 31, 224–235 (2007)CrossRefGoogle Scholar
  21. 21.
    Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)CrossRefGoogle Scholar
  22. 22.
    Russakovsky, O., Deng, J., Su, H., et al.: ImageNet large scale visual recognition challenge. arXiv:1409.0575 (2014)
  23. 23.
    Szegedy, C., Liu, W., et al.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  24. 24.
    Szegedy, C., Vanhoucke, V., Ioffe, S., et al.: Rethinking the inception architecture for computer vision. arXiv:1512.00567 (2015)
  25. 25.
    Tieleman, T., Hinton, G.: Lecture 6.5 - rmsprop, coursera. Neural networks for machine learning (2012).
  26. 26.
    U.S. Breast Cancer Statistics.
  27. 27.
    Yi, D., Sawyer, R.L., Cohn III., D., et al.: Optimizing and visualizing deep learning for benign/malignant classification in breast tumors. arXiv:1705.06362 (2017)
  28. 28.
    Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv:1605.07146 (2016)
  29. 29.
    Zhu, W., Lou, Q., Vang, Y.S., Xie, X.: Deep multi-instance networks with sparse label assignment for whole mammogram classification. arXiv:1612.05968 (2016)
  30. 30.
    Zhu, W., Xie, X.: Adversarial deep structural networks for mammographic mass segmentation. arXiv:1612.05970 (2016)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Harvard UniversityCambridgeUSA
  2. 2.DeepHealth Inc.CambridgeUSA

Personalised recommendations