Abstract
Research on explainable artificial intelligence has progressed remarkably in the last years. In the subfield of deep learning, considerable effort has been invested to the understanding of deep classifiers that have proven successful in case of various benchmark datasets. Within the methods focusing on geometry-based understanding of the trained models, an interesting, manifold disentanglement hypothesis has been proposed. This hypothesis, supported by quantitative evidence, suggests that the class distributions become gradually reorganized over the hidden layers towards lower inherent dimensionality and hence easier separability. In this work, we extend our results, concerning four datasets of low and medium complexity, and using three different assessment methods that provide robust consistent support for manifold untangling. In particular, our quantitative analysis supports the hypothesis that the data manifold becomes flattened, and the class distributions become better separable towards higher layers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For the same reason as in SVD, we did not analyse CIFAR-10 dataset.
References
Barredo Arrieta, A., et al.: Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82–115 (2020)
Brahma, P.P., Wu, D., She, Y.: Why deep learning works: a manifold disentanglement perspective. IEEE Trans. Neural Netw. Learn. Syst. 10(27), 1997–2008 (2016)
Gilmer, J., et al.: Adversarial spheres (2018). arXiv:1801.02774 [cs.CV]
Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: International Conference on Neural Information Processing, pp. 378–385 (2020)
Goodfellow, I., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: International Conference on Learning Representations (2015)
Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical report TR-2009, University of Toronto (2009)
Kuzma, T., Farkaš, I.: Embedding complexity of learned representations in neural networks. In: Tetko, I.V., Kůrková, V., Karpov, P., Theis, F. (eds.) ICANN 2019. LNCS, vol. 11728, pp. 518–528. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30484-3_42
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010). http://yann.lecun.com/exdb/mnist/
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Montavon, G., Samek, W., Müller, K.R.: Methods for interpreting and understanding deep neural networks. Digit. Signal Process. 73, 1–15 (2018)
Montúfar, G.F., Pascanu, R., Cho, K., Bengio, Y.: On the number of linear regions of deep neural networks. In: Advances in Neural Information Processing Systems, pp. 2924–2932 (2014)
Recanatesi, S., Farrell, M., Advani, M., Moore, T., Lajoie, G., Shea-Brown, E.: Dimensionality compression and expansion in deep neural networks (2019). arXiv:1906.00443 [cs.LG]
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)
Schubbach, A.: Judging machines: philosophical aspects of deep learning. Synthese (2019). https://doi.org/10.1007/s11229-019-02167-z
Schulz, A., Hinder, F., Hammer, B.: DeepView: visualizing classification boundaries of deep neural networks as scatter plots using discriminative dimensionality reduction. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence, pp. 2305–2311 (2020)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large scale image recognition. In: International Conference on Learning Representations (2015)
Stutz, D., Hein, M., Schiele, B.: Disentangling adversarial robustness and generalization (2019). arXiv:1812.00740 [cs.CV]
Szegedy, C., et al.: Intriguing properties of neural networks. In: International Conference on Learning Representations (2014)
Vilone, G., Longo, L.: Explainable artificial intelligence: a systematic review (2020). arXiv:2006.00093 [cs.AI]
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms (2017). arXiv:1708.07747 [cs.LG]
Acknowledgment
This research was partially supported by TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No 952215, and by national projects VEGA 1/0796/18 and KEGA 042UK-4/2019.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Pócoš, Š., Bečková, I., Kuzma, T., Farkaš, I. (2021). Assessment of Manifold Unfolding in Trained Deep Neural Network Classifiers. In: Heintz, F., Milano, M., O'Sullivan, B. (eds) Trustworthy AI - Integrating Learning, Optimization and Reasoning. TAILOR 2020. Lecture Notes in Computer Science(), vol 12641. Springer, Cham. https://doi.org/10.1007/978-3-030-73959-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-73959-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73958-4
Online ISBN: 978-3-030-73959-1
eBook Packages: Computer ScienceComputer Science (R0)