Abstract
Attribution methods are an easy to use tool for investigating and validating machine learning models. Multiple methods have been suggested in the literature and it is not yet clear which method is most suitable for a given task. In this study, we tested the robustness of four attribution methods, namely gradient * input, guided backpropagation, layer-wise relevance propagation and occlusion, for the task of Alzheimer’s disease classification. We have repeatedly trained a convolutional neural network (CNN) with identical training settings in order to separate structural MRI data of patients with Alzheimer’s disease and healthy controls. Afterwards, we produced attribution maps for each subject in the test data and quantitatively compared them across models and attribution methods. We show that visual comparison is not sufficient and that some widely used attribution methods produce highly inconsistent outcomes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31, pp. 9505–9515. Curran Associates, Inc. (2018), http://papers.nips.cc/paper/8160-sanity-checks-for-saliency-maps.pdf
Alvarez-Melis, D., Jaakkola, T.S.: On the robustness of interpretability methods. arXiv preprint arXiv:1806.08049 (2018)
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), 1–46 (2015). https://doi.org/10.1371/journal.pone.0130140
Bakker, R., Tiesinga, P., Kötter, R.: The scalable brain atlas: instant web-based access to public brain atlases and related content. Neuroinformatics 13(3), 353–366 (2015)
Böhle, M., Eitel, F., Weygandt, M., Ritter, K.: Layer-wise relevance propagation for explaining deep neural network decisions in MRI-based Alzheimer’s disease classification. Front. Aging Neurosci. 11, 194 (2019). https://doi.org/10.3389/fnagi.2019.00194. https://www.frontiersin.org/article/10.3389/fnagi.2019.00194
Eitel, F., et al.: Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation. CoRR (2019). http://arxiv.org/abs/1904.08771
Esmaeilzadeh, S., Belivanis, D.I., Pohl, K.M., Adeli, E.: End-to-end Alzheimer’s disease diagnosis and biomarker identification. In: Shi, Y., Suk, H.-I., Liu, M. (eds.) MLMI 2018. LNCS, vol. 11046, pp. 337–345. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_39
Korolev, S., Safiullin, A., Belyaev, M., Dodonova, Y.: Residual and plain convolutional neural networks for 3D brain MRI classification. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp. 835–838, April 2017. https://doi.org/10.1109/ISBI.2017.7950647
Liu, M., Cheng, D., Wang, K., Wang, Y.: The Alzheimer’s disease neuroimaging initiative: multi-modality cascaded convolutional neural networks for Alzheimer’s disease diagnosis. Neuroinformatics 16(3), 295–308 (2018). https://doi.org/10.1007/s12021-018-9370-4
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.R.: Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recognit. 65, 211–222 (2017). https://doi.org/10.1016/j.patcog.2016.11.008. http://www.sciencedirect.com/science/article/pii/S0031320316303582
Montavon, G., Samek, W., Müller, K.R.: Methods for interpreting and understanding deep neural networks. Digit. Signal Process. 73, 1–15 (2018). https://doi.org/10.1016/j.dsp.2017.10.011. http://www.sciencedirect.com/science/article/pii/S1051200417302385
Rieke, J., Eitel, F., Weygandt, M., Haynes, J.-D., Ritter, K.: Visualizing convolutional networks for MRI-based diagnosis of Alzheimer’s disease. In: Stoyanov, D., et al. (eds.) MLCN/DLF/IMIMIC -2018. LNCS, vol. 11038, pp. 24–31. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02628-8_3
Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 3145–3153. JMLR.org (2017)
Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)
Springenberg, J., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: the all convolutional net. In: ICLR (Workshop Track) (2015). http://lmb.informatik.uni-freiburg.de/Publications/2015/DB15a
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML 2017, pp. 3319–3328. JMLR.org (2017). http://dl.acm.org/citation.cfm?id=3305890.3306024
Vieira, S., Pinaya, W.H., Mechelli, A.: Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: methods and applications. Neurosci. Biobehav. Rev. 74, 58–75 (2017). https://doi.org/10.1016/J.NEUBIOREV.2017.01.002. https://www.sciencedirect.com/science/article/pii/S0149763416305176
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Funding
We acknowledge support from the German Research Foundation (DFG, 389563835), the Manfred and Ursula-Müller Stiftung, the Brain & Behavior Research Foundation (NARSAD grant, USA), the Deutsche Multiple Sklerose Gesellschaft (DMSG) Bundesverband e.V. and Charité – Universitätsmedizin Berlin (Rahel-Hirsch scholarship).
Author information
Authors and Affiliations
Consortia
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Eitel, F., Ritter, K., for the Alzheimer’s Disease Neuroimaging Initiative (ADNI). (2019). Testing the Robustness of Attribution Methods for Convolutional Neural Networks in MRI-Based Alzheimer’s Disease Classification. In: Suzuki, K., et al. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support. ML-CDS IMIMIC 2019 2019. Lecture Notes in Computer Science(), vol 11797. Springer, Cham. https://doi.org/10.1007/978-3-030-33850-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-33850-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33849-7
Online ISBN: 978-3-030-33850-3
eBook Packages: Computer ScienceComputer Science (R0)
