Hybrid Mass Detection in Breast MRI Combining Unsupervised Saliency Analysis and Deep Learning

  • Guy AmitEmail author
  • Omer Hadad
  • Sharon Alpert
  • Tal Tlusty
  • Yaniv Gur
  • Rami Ben-Ari
  • Sharbell Hashoul
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10435)


To interpret a breast MRI study, a radiologist has to examine over 1000 images, and integrate spatial and temporal information from multiple sequences. The automated detection and classification of suspicious lesions can help reduce the workload and improve accuracy. We describe a hybrid mass-detection algorithm that combines unsupervised candidate detection with deep learning-based classification. The detection algorithm first identifies image-salient regions, as well as regions that are cross-salient with respect to the contralateral breast image. We then use a convolutional neural network (CNN) to classify the detected candidates into true-positive and false-positive masses. The network uses a novel multi-channel image representation; this representation encompasses information from the anatomical and kinetic image features, as well as saliency maps. We evaluated our algorithm on a dataset of MRI studies from 171 patients, with 1957 annotated slices of malignant (59%) and benign (41%) masses. Unsupervised saliency-based detection provided a sensitivity of 0.96 with 9.7 false-positive detections per slice. Combined with CNN classification, the number of false positive detections dropped to 0.7 per slice, with 0.85 sensitivity. The multi-channel representation achieved higher classification performance compared to single-channel images. The combination of domain-specific unsupervised methods and general-purpose supervised learning offers advantages for medical imaging applications, and may improve the ability of automated algorithms to assist radiologists.


Breast MRI Lesion detection Saliency Deep learning 


  1. 1.
    Vignati, A., Giannini, V., et al.: A fully automatic lesion detection method for DCE-MRI fat-suppressed breast images, 26 February 2009Google Scholar
  2. 2.
    Ertas, G., Doran, S., Leach, M.O.: Computerized detection of breast lesions in multi-centre and multi-instrument DCE-MR data using 3D principal component maps and template matching. Phys. Med. Biol. 56, 7795–7811 (2011)CrossRefGoogle Scholar
  3. 3.
    McClymont, D., Mehnert, A., et al.: Fully automatic lesion segmentation in breast MRI using mean-shift and graph-cuts on a region adjacency graph. J. Magn. Reson. Imaging 39, 795–804 (2014)CrossRefGoogle Scholar
  4. 4.
    Gallego-Ortiz, C., Martel, A.L.: Improving the accuracy of computer-aided diagnosis for breast MR imaging by differentiating between mass and nonmass lesions. Radiology 278, 679–688 (2016)CrossRefGoogle Scholar
  5. 5.
    Agner, S.C., Soman, S., et al.: Textural kinetics: a novel dynamic contrast-enhanced (DCE)-MRI feature for breast lesion classification. J. Digit. Imaging 24, 446–463 (2011)CrossRefGoogle Scholar
  6. 6.
    Ertaş, G., Gülçür, H.Ö., et al.: Breast MR segmentation and lesion detection with cellular neural networks and 3D template matching. Comput. Biol. Med. 38, 116–126 (2008)CrossRefGoogle Scholar
  7. 7.
    Renz, D.M., Böttcher, J., et al.: Detection and classification of contrast-enhancing masses by a fully automatic computer-assisted diagnosis system for breast MRI. J. Magn. Reson. Imaging 35, 1077–1088 (2012)CrossRefGoogle Scholar
  8. 8.
    Pang, Z., Zhu, D., Chen, D., Li, L., Shao, Y.: A computer-aided diagnosis system for dynamic contrast-enhanced MR images based on level set segmentation and ReliefF feature selection. Comput. Math. Methods Med. 2015(2015), Article ID 450531 (2015). doi: 10.1155/2015/450531 CrossRefGoogle Scholar
  9. 9.
    Amit, G., Ben-Ari, R., Hadad, O., Monovich, E., Granot, N., Hashoul, S.: Classification of breast MRI lesions using small-size training sets: comparison of deep learning approaches. In: Proceedings of SPIE Medical Imaging, vol. 10134 (2017)Google Scholar
  10. 10.
    Wu, H., Cristina Gallego-Ortiz, A.M.: Deep artificial neural network approach to automated lesion segmentation in breast DCE-MRI. In: Proceedings of the 3rd MICCAI Workshop on Breast Image Analysis, pp. 73–80 (2015)Google Scholar
  11. 11.
    Erihov, M., Alpert, S., Kisilev, P., Hashoul, S.: A cross saliency approach to asymmetry-based tumor detection (2015)Google Scholar
  12. 12.
    Clark, K., Vendt, B., et al.: The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging. 26, 1045–1057 (2013)CrossRefGoogle Scholar
  13. 13.
    Lingle, W., Erickson, B.J., et al.: Radiology Data from The Cancer Genome Atlas Breast Invasive Carcinoma [TCGA-BRCA] collection.
  14. 14.
    Bloch, B.N., Jain, A., Jaffe, C.C.: Data from breast-diagnosis.
  15. 15.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  16. 16.
    Margolin, R., Tal, A., Zelnik-Manor, L.: What makes a patch distinct? In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1139–1146. IEEE (2013)Google Scholar
  17. 17.
    Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A.: The generalized PatchMatch correspondence algorithm. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6313, pp. 29–43. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-15558-1_3 CrossRefGoogle Scholar
  18. 18.
    Hadad, O., Bakalo, R., Ben-Ar, R., Hashoul, S., Amit, G.: Classification of breast lesions using cross-modal deep learning. In: IEEE International Symposium on Biomedical Imaging (ISBI) (2017)Google Scholar
  19. 19.
    Lehman, C.D., Blume, J.D., et al.: Accuracy and interpretation time of computer-aided detection among novice and experienced breast MRI readers. Am. J. Roentgenol. 200, W683–W689 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Guy Amit
    • 1
    Email author
  • Omer Hadad
    • 1
  • Sharon Alpert
    • 1
  • Tal Tlusty
    • 1
  • Yaniv Gur
    • 1
  • Rami Ben-Ari
    • 1
  • Sharbell Hashoul
    • 1
  1. 1.Haifa University CampusMount Carmel HaifaIsrael

Personalised recommendations