Abstract
Convolutional neural networks have shown outstanding object recognition performance, especially for visual recognition tasks such as tumor classification in 2D ultrasound (US) images. In Computer-Aided Diagnosis (CAD) systems, interpreting CNN’s decision is crucial for accepting the system in the clinical use. This paper is concerned with ‘visual explanations’ for decisions from CNN models trained on ultrasound images. In particular, we investigate the link between the CNN decision and the calcification cancer sign in breast lesion classification task. To this end, we study the output visualization of two different breast lesion recognition CNN models in two folds: Firstly, we explore two existing visualization approaches, Grad-CAM and CRM, to gain insight into the function of feature layers. Secondly, we introduce an adaptive Grad-CAM, called EGrad-CAM, which uses information entropy to freeze feature maps with no or minimal information. Extensive analysis and experiments using 1624 US images and two breast classification models show that calcification feature contributes to the CNN classification decision for both malignant and benign lesions. Furthermore, we show many feature maps in the final convolution layer are not contributing to the CNN decision, and our EGrad-CAM produces similar visualization output to Grad-CAM using 24%–87% of the feature maps. Our study demonstrates that the CNN decision visualization is a promising direction for bridging the gap between CNN classification decision of US images of breast lesions and cancer signs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R.L., Torre, L.A., Jemal, A.: Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: Cancer J. Clin. 68(6), 394–424 (2018). https://doi.org/10.3322/caac.21492
Mercado, C.L.: Bi-rads update. Radiol. Clin. 52(3), 481–487 (2014)
Zhu, Y.-C., et al.: A generic deep learning framework to classify thyroid and breast lesions in ultrasound images. Ultrasonics 110, 106300 (2021)
Wang, Y., Choi, E.J., Choi, Y., Zhang, H., Jin, G.Y., Ko, S.-B.: Breast cancer classification in automated breast ultrasound using multiview convolutional neural network with transfer learning. Ultrasound Med. Biol. 46(5), 1119–1132 (2020)
Tanaka, H., Chiu, S.-W., Watanabe, T., Kaoku, S., Yamaguchi, T.: Computer-aided diagnosis system for breast ultrasound images using deep learning. Phys. Med. Biol. 64(23), 235013 (2019)
Moon, W.K., Lee, Y.-W., Ke, H.-H., Lee, S.H., Huang, C.-S., Chang, R.-F.: Computer‐aided diagnosis of breast ultrasound images using ensemble learning from convolutional neural networks. Comput. Meth. Program. Biomed. 190, 105361 (2020)
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)
Kim, I., Rajaraman, S., Antani, S.: Visual interpretation of convolutional neural network predictions in classifying medical image modalities. Diagnostics 9(2), 38 (2019)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., Alsaadi, F.E.: A survey of deep neural network architectures and their applications. Neurocomputing 234, 11–26 (2017)
Byra, M., et al.: Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion. Med. Phys. 46(2), 746–755 (2019)
Rony, J., Belharbi, S., Dolz, J., Ayed, I.B., McCaffrey, L., Granger, E.: Deep weakly-supervised learning methods for classification and localization in histology images: a survey. arXiv preprint arXiv:1909.03354 (2019)
Wang, H., et al.: Score-CAM: score-weighted visual explanations for convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 24–25 (2020)
Fukui, H., Hirakawa, T., Yamashita, T., Fujiyoshi, H.: Attention branch network: learning of attention mechanism for visual explanation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10705–10714 (2019)
Zhou, L.-Q., et al.: Lymph node metastasis prediction from primary breast cancer US images using deep learning. Radiology 294(1), 19–28 (2020)
Xie, B., et al.: Computer-aided diagnosis for fetal brain ultrasound images using deep convolutional neural networks. Int. J. Comput. Assist. Radiol. Surg. 15(8), 1303–1312 (2020). https://doi.org/10.1007/s11548-020-02182-3
Reyes, M., et al.: On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiol. Artif. Intell. 2(3), e190043 (2020). https://doi.org/10.1148/ryai.2020190043
Yang, H., Kim, J.-Y., Kim, H., Adhikari, S.P.: Guided soft attention network for classification of breast cancer histopathology images. IEEE Trans. Med. Imaging 39(5), 1306–1315 (2019)
Schlemper, J., et al.: Attention gated networks: learning to leverage salient regions in medical images. Med. Image Anal. 53, 197–207 (2019)
Toussaint, N., et al.: Weakly supervised localisation for fetal ultrasound images. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 192–200. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_22
Baumgartner, C.F., Koch, L.M., Tezcan, K.C., Ang, J.X., Konukoglu, E.: Visual feature attribution using Wasserstein GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8309–8319 (2018)
Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948)
Acknowledgments
This research is sponsored by TenD Innovations.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Eskandari, A., Du, H., AlZoubi, A. (2021). Towards Linking CNN Decisions with Cancer Signs for Breast Lesion Classification from Ultrasound Images. In: Papież, B.W., Yaqub, M., Jiao, J., Namburete, A.I.L., Noble, J.A. (eds) Medical Image Understanding and Analysis. MIUA 2021. Lecture Notes in Computer Science(), vol 12722. Springer, Cham. https://doi.org/10.1007/978-3-030-80432-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-80432-9_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80431-2
Online ISBN: 978-3-030-80432-9
eBook Packages: Computer ScienceComputer Science (R0)