Classification of contrast-enhanced spectral mammography (CESM) images



Contrast-enhanced spectral mammography (CESM) is a recently developed breast imaging technique. CESM relies on dual-energy acquisition following contrast agent injection to improve mammography sensitivity. CESM is comparable to contrast-enhanced MRI in terms of sensitivity, at a fraction of the cost. However, since lesion variability is large, even with the improved visibility provided by CESM, differentiation between benign and malignant enhancement is not accurate and a biopsy is usually performed for final assessment. Breast biopsies can be stressful to the patient and are expensive to healthcare systems. Moreover, as the biopsies results are most of the time benign, a specificity improvement in the radiologist diagnosis is required. This work presents a deep learning-based decision support system, which aims at improving the specificity of breast cancer diagnosis by CESM without affecting sensitivity.


We compare two analysis approaches, fine-tuning a pretrained network and fully training a convolutional neural network, for classification of CESM breast mass as benign or malignant. Breast Imaging Reporting and Data Systems (BIRADS) is a radiological lexicon, used with breast images, to categorize lesions. We improve each classification network by incorporating BIRADS textual features as an additional input to the network. We evaluate two ways of BIRADS fusion as network input: feature fusion and decision fusion. This leads to multimodal network architectures. At classification, we also exploit information from apparently normal breast tissue in the CESM of the considered patient, leading to a patient-specific classification.


We evaluate performance using fivefold cross-validation, on 129 randomly selected breast lesions annotated by an experienced radiologist. Each annotation includes a contour of the mass in the image, biopsy-proven label of benign or malignant lesion and BIRADS descriptors. At 100% sensitivity, specificity of 66% was achieved using a multimodal network, which combines inputs at feature level and patient-specific classification.


The presented multimodal network may significantly reduce benign biopsies, without compromising sensitivity.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6


  1. 1.

    Carney PA, Miglioretti DL, Yankaskas BC, Kerlikowske K, Rosenberg R, Rutter CM, Geller BM, Abraham LA, Taplin SH, Dignan M, Cutter G (2003) Individual and combined effects of age, breast density, and hormone replacement therapy use on the accuracy of screening mammography. Ann Intern Med 138(3):168–75

    PubMed  Article  Google Scholar 

  2. 2.

    Drukteinis JS, Mooney BP, Flowers CI, Gatenby RA (2013) Beyond mammography: new frontiers in breast cancer screening. Am J Med 126(6):472–479

    PubMed  PubMed Central  Article  Google Scholar 

  3. 3.

    Lobbes MBI, Smidt ML, Houwers J, Tjan-Heijnen VC, Wildberger JE (2013) Contrast enhanced mammography: techniques, current results, and potential indications. Clin Radiol 68(9):935–944

    CAS  PubMed  Article  Google Scholar 

  4. 4.

    Li L, Roth R, Germaine P, Ren S, Lee M, Hunter K, Tinney E, Liao L (2017) Contrast-enhanced spectral mammography (CESM) versus breast magnetic resonance imaging (MRI): a retrospective comparison in 66 breast lesions. Diagn Interv Imaging 98:113123

    Google Scholar 

  5. 5.

    Fallenberg EM, Dromain C, Diekmann F, Engelken F, Krohn M, Singh JM, Ingold-Heppner B, Winzer KJ, Bick U, Renz AD (2014) Contrast-enhanced spectral mammography versus MRI: initial results in the detection of breast cancer and assessment of tumour size. Eur Radiol 24(1):256–264

    CAS  PubMed  Article  Google Scholar 

  6. 6.

    Luczyska E, Heinze-Paluchowska S, Dyczek S, Blecharz P, Rys J, Reinfuss M (2014) Contrast-enhanced spectral mammography: comparison with conventional mammography and histopathology in 152 women. Korean J Radiol 15(6):689–696

    Article  Google Scholar 

  7. 7.

    Brake GM, Karssemeijer N, Hendriks JH (2000) An automatic method to discriminate malignant masses from normal tissue in digital mammograms1. Phys Med Biol 45(10):2843

    Article  Google Scholar 

  8. 8.

    Sampat MP, Markey MK, Bovik AC (2005) Computer-aided detection and diagnosis in mammography. Handb Image Video Process 2(1):1195–1217

    Article  Google Scholar 

  9. 9.

    Vasantha M, Bharathi DVS, Dhamodharan R (2010) Medical image feature, extraction, selection and classification. IJEST 1(2):2071–2076

    Google Scholar 

  10. 10.

    Jalalian A, Mashohor SB, Mahmud HR, Saripan MIB, Ramli ARB, Karasfi B (2013) Computer-aided detection/diagnosis of breast cancer in mammography and ultrasound: a review. Clin Imaging 37(3):420–426

    PubMed  Article  Google Scholar 

  11. 11.

    Mateos MJ, Gastelum A, Mrquez J, Brandan ME (2016) Texture analysis of contrast-enhanced digital mammography (CEDM) images. International Workshop on Digital Mammography. LNCS 9699, Springer, pp 585–592

    Google Scholar 

  12. 12.

    Cheung YC, Tsai HP, Lo YF, Ueng SH, Huang PC, Chen SC (2016) Clinical utility of dual-energy contrast-enhanced spectral mammography for breast microcalcifications without associated mass: a preliminary analysis. Eur Radiol 26(4):1082–1089

    PubMed  Article  Google Scholar 

  13. 13.

    Patel BK, Ranjbar S, Wu T, Pockaj BA, Li J, Zhang N, Lobbes M, Zhang B, Mitchell JR (2018) Computer-aided diagnosis of contrast-enhanced spectral mammography: a feasibility study. Eur J Radiol 98:207–213

    PubMed  Article  Google Scholar 

  14. 14.

    Pudil P, Novoviov J, Kittler J (1994) Floating search methods in feature selection. Pattern Recognit Lett 15(11):1119–1125

    Article  Google Scholar 

  15. 15.

    Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    Google Scholar 

  16. 16.

    American College of Radiology, ACR BI-RADS Atlas 5th Edition, 125–143 (2013)

  17. 17.

    Knogler T, Homolka P, Hoernig M, Leithner R, Langs G, Waitzbauer M, Pinker-Domenig K, Leitner S, Helbich TH (2017) Application of BI-RADS descriptors in contrast-enhanced dual-energy mammography: comparison with MRI. Breast Care 12:212–216

    PubMed  PubMed Central  Article  Google Scholar 

  18. 18.

    Ngiam J, Khosla A, Kim M, Nam J, Lee H, Ng AY (2011) Multimodal deep learning. In: Proceedings of the 28th ICML

  19. 19.

    Wu D, Pigou L (2016) Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1583–1597

    PubMed  Article  Google Scholar 

  20. 20.

    Cao Y, Steffey S, He J, Xiao D, Tao C, Chen P, Mller H (2014) Medical image retrieval: a multimodal approach. Cancer Inf 13:CIN-S14053

    Article  Google Scholar 

  21. 21.

    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. NIPS 25:1097–1105

    Google Scholar 

  22. 22.

    Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE TMI 35(5):1299–1312

    Google Scholar 

  23. 23.

    Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L (2014) Large-scale video classification with convolutional neural networks. In: IEEE conference CVPR, pp 1725–1732

  24. 24.

    Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: ICML, pp 448–456

  25. 25.

    Kingma D, Adam JB (2014) A method for stochastic optimization. arXiv preprint arXiv:1412.6980

Download references

Author information



Corresponding author

Correspondence to Shaked Perek.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

For this type of study formal consent is not required.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Perek, S., Kiryati, N., Zimmerman-Moreno, G. et al. Classification of contrast-enhanced spectral mammography (CESM) images. Int J CARS 14, 249–257 (2019).

Download citation


  • Deep learning
  • Contrast-enhanced spectral mammography (CESM)
  • Breast cancer
  • Multimodal neural networks
  • Computer vision