Abstract
In recent years, the interpretive this looks like that structure has gained significant attention. It refers to the human tendency to break down images into key parts and make classification decisions by comparing them to pre-existing concepts in their minds. However, most existing prototypical-based models assign prototypes directly to each category without considering that key parts with the same meaning may appear in images from different categories. To address this issue, we propose dividing prototypes with the same meaning into the same latent space (referred to as Basic Feature Domain) since different category parts only slightly affect the corresponding prototype vectors. This process of integrating prototypes based on the feature domain is referred to as prototype alignment. Additionally, we introduce the concept of part-aware optimization, which prioritizes prototypical parts of images over simple category labels during optimizing prototypes. Moreover, we present two feature aggregation methods, by row and by cluster, for the basic feature domain. We demonstrate competitive results compared to other state-of-the-art prototypical part methods on the CUB-2011-200 dataset and Stanford Cars dataset using our proposed self-explanatory part-aware proto-aligned network (PaProtoPNet).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Afnan, M.A.M., et al.: Interpretable, not black-box, artificial intelligence should be used for embryo selection. Hum. Reprod. Open 2021(4), hoab040 (2021)
Donnelly, J., Barnett, A.J., Chen, C.: Deformable ProtoPNet: an interpretable image classifier using deformable prototypes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10265–10275 (2022)
Ghorbani, A., Wexler, J., Zou, J.Y., Kim, B.: This looks like that: deep learning for interpretable image recognition. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Ghorbani, A., Wexler, J., Zou, J.Y., Kim, B.: Towards automatic concept-based explanations. In: Advances in Neural Information Processing Systems, pp. 9273–9282 (2019)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Kim, B., et al.: Interpretability beyond feature attribution: quantitative testing with concept activation vectors (TCAV). In: Proceedings of Machine Learning Research, vol. 80, pp. 2668–2677 (2018)
Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine-grained categorization. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 554–561 (2013)
Nauta, M., Van Bree, R., Seifert, C.: Neural prototype trees for interpretable fine-grained image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14933–14943 (2021)
Rymarczyk, D., Struski, Ł, Górszczak, M., Lewandowska, K., Tabor, J., Zieliński, B.: Interpretable Image Classification with Differentiable Prototypes Assignment. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13672, pp. 351–368. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19775-8_21
Shafiq, M., Gu, Z.: Deep residual learning for image recognition: a survey. Appl. Sci. 12(18), 1–43 (2022)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Ukai, Y., Hirakawa, T., Yamashita, T., Fujiyoshi, H.: This looks like it rather than that: ProtoKNN for similarity-based classifiers. In: The Eleventh International Conference on Learning Representations, pp. 1–17 (2023)
Van Horn, G., et al.: The inaturalist species classification and detection dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8769–8778 (2018)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset (2011)
Wang, J., Liu, H., Wang, X., Jing, L.: Interpretable image recognition by constructing transparent embedding space. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 895–904 (2021)
Wiegand, G., Schmidmaier, M., Weber, T., Liu, Y., Hussmann, H.: I drive-you trust: explaining driving behavior of autonomous cars. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–6 (2019)
Zhang, R., Madumal, P., Miller, T., Ehinger, K.A., Rubinstein, B.I.: Invertible concept-based explanations for CNN models with non-negative concept activation vectors. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 11682–11690 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, L., Gong, X., Wang, C., Kong, W. (2024). Part-Aware Prototype-Aligned Interpretable Image Classification with Basic Feature Domain. In: Liu, T., Webb, G., Yue, L., Wang, D. (eds) AI 2023: Advances in Artificial Intelligence. AI 2023. Lecture Notes in Computer Science(), vol 14472. Springer, Singapore. https://doi.org/10.1007/978-981-99-8391-9_15
Download citation
DOI: https://doi.org/10.1007/978-981-99-8391-9_15
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8390-2
Online ISBN: 978-981-99-8391-9
eBook Packages: Computer ScienceComputer Science (R0)