Abstract
The recent success of deep learning models in solving complex problems and in different domains has increased interest in understanding what they learn. Therefore, different approaches have been employed to explain these models, one of which uses human-understandable concepts as explanations. Two examples of methods that use this approach are Network Dissection [5] and Compositional explanations [23]. The former explains units using atomic concepts, while the latter makes explanations more expressive, replacing atomic concepts with logical forms. While intuitively, logical forms are more informative than atomic concepts, it is not clear how to quantify this improvement, and their evaluation is often based on the same metric that is optimized during the search-process and on the usage of hyper-parameters to be tuned. In this paper, we propose to use as evaluation metric the Detection Accuracy, which measures units’ consistency of detection of their assigned explanations. We show that this metric (1) evaluates explanations of different lengths effectively, (2) can be used as a stopping criterion for the compositional explanation search, eliminating the explanation length hyper-parameter, and (3) exposes new specialized units whose length 1 explanations are the perceptual abstractions of their longer explanations. Code available at https://github.com/KRLGroup/detacc-compexp.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adebayo, J., Gilmer, J., Muelly, M. Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps (2018). http://arxiv.org/abs/1810.03292
Alqahtani, A., Xie, X., Jones, M.W., Essa, E.: Pruning CNN filters via quantifying the importance of deep visual representations. Comput. Vis. Image Underst. 208, 103220 (2021). ISSN 1077–3142. https://doi.org/10.1016/j.cviu.2021.103220, https://www.sciencedirect.com/science/article/pii/S1077314221000643
Ancona, M., Ceolini, E., Öztireli, A.C., Gross, M.H.: A unified view of gradient-based attribution methods for deep neural networks (2017). http://arxiv.org/abs/1711.06104
Arrieta, A.B., et al.: Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82–115 (2020). ISSN 1566–2535. https://doi.org/10.1016/j.inffus.2019.12.012, https://www.sciencedirect.com/science/article/pii/S1566253519308103
Bau, D., Zhou, B., Khosla, A., Oliva, A., Torralba, A.: Network dissection: quantifying interpretability of deep visual representations (2017). http://arxiv.org/abs/1704.05796
Dalvi, F., et al.: Neurox: a toolkit for analyzing individual neurons in neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 9851–9852 (2019). https://doi.org/10.1609/aaai.v33i01.33019851, https://ojs.aaai.org/index.php/AAAI/article/view/5063
Došilović, F.K., Brčić, M., Hlupić, N.: Explainable artificial intelligence: a survey. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 0210–0215 (2018). https://doi.org/10.23919/MIPRO.2018.8400040
Ghorbani, A., Wexler, J., Zou, J., Kim, B.: Towards automatic concept-based explanations (2019)
Gonzalez-Garcia, A., Modolo, D., Ferrari, V.: Do semantic parts emerge in convolutional neural networks? (2016). http://arxiv.org/abs/1607.03738
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Hilton, D.J.: Mental models and causal explanation: judgements of probable cause and explanatory relevance. Thinking Reasoning 2(4), 273–308 (1996)
Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks (2016). http://arxiv.org/abs/1608.06993
Kim, B., Koyejo, O., Khanna, R.: Examples are not enough, learn to criticize! criticism for interpretability. In: Lee, D.D., Sugiyama, M., von Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 5–10 December 2016, Barcelona, Spain, pp. 2280–2288 (2016). https://proceedings.neurips.cc/paper/2016/hash/5680522b8e2bb01943234bce7bf84534-Abstract.html
Kim, B., et al.: Interpretability beyond feature attribution: quantitative Testing with Concept Activation Vectors (TCAV) (2018)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25. Curran Associates Inc (2012). https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
Le, Q.V., et al.: Building high-level features using large scale unsupervised learning (2011). http://arxiv.org/abs/1112.6209
Lin, Y., Lee, W., Celik, Z.B.: What do you see? evaluation of explainable artificial intelligence (XAI) interpretability through neural backdoors (2020). https://arxiv.org/abs/2009.10639
Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates Inc (2017). https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf
Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them (2014). http://arxiv.org/abs/1412.0035
Miller, T.: Explanation in artificial intelligence: insights from the social sciences (2017). http://arxiv.org/abs/1706.07269
Mittelstadt, B.D., Russell, C., Wachter, S.: Explaining explanations in AI (2018). http://arxiv.org/abs/1811.01439
Molnar, C.: Interpretable machine learning (2019). https://christophm.github.io/interpretable-ml-book/
Mu, J., Andreas, J.: Compositional explanations of neurons (2020). https://arxiv.org/abs/2006.14032
Nguyen, A.M., Dosovitskiy, A., Yosinski, J., Brox, T., Clune, J.: Synthesizing the preferred inputs for neurons in neural networks via deep generator networks (2016). http://arxiv.org/abs/1605.09304
Rosenfeld, A.: Better metrics for evaluating explainable artificial intelligence. In: International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, pp. 45–50 (2021). 9781450383073
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., Müller, K.-R.: Evaluating the visualization of what a deep neural network has learned. IEEE Trans. Neural Networks Learn. Syst. 28(11), 2660–2673 (2017). https://doi.org/10.1109/TNNLS.2016.2599820
Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps (2014)
Slugoski, B.R., Lalljee, M., Lamb, R., Ginsburg, G.P.: Attribution in conversational context: effect of mutual knowledge on explanation-giving. Eur. J. Soc. Psychol. 23(3), 219–238 (1993)
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 70, pp. 3319–3328. PMLR (2017). http://proceedings.mlr.press/v70/sundararajan17a.html
Vondrick, C., Pirsiavash, H., Torralba, A.: Generating videos with scene dynamics (2016). http://arxiv.org/abs/1609.02612
Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., Zhu, J.: Explainable AI: a brief survey on history, research areas, approaches and challenges. In: Tang, J., Kan, M.-Y., Zhao, D., Li, S., Zan, H. (eds.) Natural Language Processing and Chinese Computing, pp. 563–574 (2019). Springer International Publishing, Cham. ISBN 978-3-030-32236-6
Yang, F., Du, M., Hu, X.: Evaluating explanation without ground truth in interpretable machine learning (2019) http://arxiv.org/abs/1907.06831
Yeh, C., Hsieh, C., Suggala, A.S., Inouye, D.I., Ravikumar, P.: How sensitive are sensitivity-based explanations? (2019). http://arxiv.org/abs/1901.09392
Yeom, S., Seegerer, P., Lapuschkin, S., Wiedemann, S., Müller, K., Samek, W.: Pruning by explaining: a novel criterion for deep neural network pruning (2019). http://arxiv.org/abs/1912.08881
Ylikoski, P.: Causal and constitutive explanation compared. Erkenntnis 78(2), 277–297 (2013). https://doi.org/10.1007/s10670-013-9513-9
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks (2013). http://arxiv.org/abs/1311.2901
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Object detectors emerge in deep scene CNNS (2015)
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ADE20K dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5122–5130 (2017). https://doi.org/10.1109/CVPR.2017.544
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018). https://doi.org/10.1109/TPAMI.2017.2723009
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Makinwa, S.M., La Rosa, B., Capobianco, R. (2022). Detection Accuracy for Evaluating Compositional Explanations of Units. In: Bandini, S., Gasparini, F., Mascardi, V., Palmonari, M., Vizzari, G. (eds) AIxIA 2021 – Advances in Artificial Intelligence. AIxIA 2021. Lecture Notes in Computer Science(), vol 13196. Springer, Cham. https://doi.org/10.1007/978-3-031-08421-8_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-08421-8_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08420-1
Online ISBN: 978-3-031-08421-8
eBook Packages: Computer ScienceComputer Science (R0)