Abstract
Rapid progress in machine learning and artificial intelligence (AI) has brought increased attention to the potential security and reliability of AI technologies. This paper identifies the threat of network incorrectly relying on counterfactual features that can stay undetectable during validation but cause serious issues in life application. Furthermore, we propose a method to counter this hazard. It combines well-known techniques: object detection tool and saliency map obtaining formula to compute metric indicating potentially faulty learning. We prove the effectiveness of the method, as well as discuss its shortcomings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)
Vung, P., Pham, C., Dang, T.: Road damage detection and classification with detectron2 and faster R-CNN. arXiv preprint arXiv:2010.15021 (2020)
Chattopadhay, A., et al.: Grad-cam++: generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Dinh, V., Ho, L.S.T.: Consistent feature selection for analytic deep neural networks. arXiv preprint arXiv:2010.08097 (2020)
Detectron2, Detectron2 modelzoo. https://github.com/facebookresearch/detectron2/ Accessed 16 Oct 2020
Zhang, J., Bargal, S.A., Lin, Z., Brandt, J., Shen, X., Sclaroff, S.: Top-down neural attention by excitation backprop. Int. J. Comput. Vis. 126(10), 1084–1102 (2017). https://doi.org/10.1007/s11263-017-1059-x
Shokoufandeh, A., Marsic, I., Dickinson, S.J.: View-based object recognition using saliency maps. Image Vis. Comput. 17(5–6), 445–460 (1999)
Deng, Dong, W. Socher, R., Li, L.-J., Li, K., Li, F.-F.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)
Engstrom, L., et al.: Adversarial robustness as a prior for learned representations. arXiv preprint arXiv:1906.00945 (2019)
Schott, L., et al.: Towards the first adversarially robust neural network model on MNIST. arXiv preprint arXiv:1805.09190 (2018)
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you?: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2016)
Lapuschkin, S., et al.: Unmasking Clever Hans predictors and assessing what machines really learn. Nat. Commun. 10(1), 1096 (2019)
Zech, J.R., et al.: Confounding variables can degrade generalization performance of radiological deep learning models. arXiv preprint arXiv:1807.00431 (2018)
Xiao, K., et al.: Noise or signal: the role of image backgrounds in object recognition. arXiv preprint arXiv:2006.09994 (2020)
Zhang, J., et al.: Local features and kernels for classification of texture and object categories: a comprehensive study. Int. J. Comput. Vis. 73(2), 213–238 (2007)
Zhu, Z., Xie, L., Yuille, A.L.: Object recognition with and without objects. arXiv preprint arXiv:1611.06596 (2016)
Sagawa, S., et al.: Distributionally robust neural networks for group shifts: on the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 (2019)
Torralba, A.: Contextual priming for object detection. Int. J. Comput. Vis. 53(2), 169–191 (2003)
Geirhos, R., et al.: ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Szandała, T., Maciejewski, H. (2021). Automated Method for Evaluating Neural Network’s Attention Focus. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2021. ICCS 2021. Lecture Notes in Computer Science(), vol 12743. Springer, Cham. https://doi.org/10.1007/978-3-030-77964-1_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-77964-1_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-77963-4
Online ISBN: 978-3-030-77964-1
eBook Packages: Computer ScienceComputer Science (R0)