Abstract
Comparing with traditional convolutional neural networks, the recent capsule networks based on routing-by-agreement have shown their robustness and interpretability. However, the routing mechanisms lead to huge computations and parameters, significantly reducing the effectiveness and efficiency of capsule networks towards complex and large-scale data. Moreover, the capsule network and its variants only explore the PrimaryCaps layer for various reasoning tasks while ignoring the specific information learned by the low-level convolutional layers. In this paper, we propose a Graph Routing based on Multi-head Pairwise-relation Attention (GraMPA) that could thoroughly exploit both the semantic and location similarity of capsules in the form of multi-head graphs, to improve the robustness of the capsule network with much fewer parameters. Moreover, a Global Fusion Capsule Network (GFCN) architecture based on the Multi-block Attention (MBA) module and Multi-block Feature Fusion (MBFF) module is designed, aiming to fully explore detailed low-level signals and global information to enhance the quality of final representations especially for complicated images. Exhaustive experiments show that our method can achieve better classification performance and robustness on CIFAR10 and SVHN datasets with fewer parameters and computations. The source code of GraMPA is available at https://github.com/SWU-CS-MediaLab/GraMPA.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmed, K., Torresani, L.: Star-Caps: capsule networks with straight-through attentive routing. In: NeurIPS, pp. 9098–9107 (2019)
Cai, S., Zuo, W., Zhang, L.: Higher-order integration of hierarchical convolutional activations for fine-grained visual categorization. In: ICCV, pp. 511–520. IEEE Computer Society (2017)
Cao, Y., Xu, J., Lin, S., Wei, F., Hu, H.: GCNet: non-local networks meet squeeze-excitation networks and beyond. In: ICCV Workshops, pp. 1971–1980. IEEE (2019)
Chang, X., Hospedales, T.M., Xiang, T.: Multi-level factorisation net for person re-identification. In: CVPR, pp. 2109–2118. IEEE Computer Society (2018)
Durand, T., Mordan, T., Thome, N., Cord, M.: WILDCAT: weakly supervised learning of deep convnets for image classification, pointwise localization and segmentation. In: CVPR, pp. 5957–5966. IEEE Computer Society (2017)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: ICLR (Poster) (2015)
Gu, J., Tresp, V.: Interpretable graph capsule networks for object recognition. CoRR abs/2012.01674 (2020)
Hahn, T., Pyeon, M., Kim, G.: Self-routing capsule networks. In: NeurIPS, pp. 7656–7665 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778. IEEE Computer Society (2016)
Hinton, G.E., Sabour, S., Frosst, N.: Matrix capsules with EM routing. In: ICLR (Poster). OpenReview.net (2018)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011)
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: NIPS, pp. 3856–3866 (2017)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: ICCV, pp. 618–626. IEEE Computer Society (2017)
Szegedy, C., et al.: Intriguing properties of neural networks. In: ICLR (Poster) (2014)
Tsai, Y.H., Srivastava, N., Goh, H., Salakhutdinov, R.: Capsules with inverted dot-product attention routing. In: ICLR. OpenReview.net (2020)
Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)
Verma, S., Zhang, Z.: Graph capsule convolutional neural networks. CoRR abs/1805.08090 (2018)
Wang, H., Wang, S., Qin, Z., Zhang, Y., Li, R., Xia, Y.: Triple attention learning for classification of 14 thoracic diseases using chest radiography. Med. Image Anal. 67, 101846 (2021)
Wang, X., Girshick, R.B., Gupta, A., He, K.: Non-local neural networks. In: CVPR, pp. 7794–7803. IEEE Computer Society (2018)
Wang, X., Zou, X., Bakker, E.M., Wu, S.: Self-constraining and attention-based hashing network for bit-scalable cross-modal retrieval. Neurocomputing 400, 255–271 (2020). https://doi.org/10.1016/j.neucom.2020.03.019. https://www.sciencedirect.com/science/article/pii/S0925231220303544
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1
Wu, S., Oerlemans, A., Bakker, E.M., Lew, M.S.: Deep binary codes for large scale image retrieval. Neurocomputing 257, 5–15 (2017). Machine Learning and Signal Processing for Big Multimedia Analysis. https://doi.org/10.1016/j.neucom.2016.12.070. https://www.sciencedirect.com/science/article/pii/S0925231217301455
Xinyi, Z., Chen, L.: Capsule graph neural network. In: ICLR (Poster), OpenReview.net (2019)
Yang, S., Ramanan, D.: Multi-scale recognition with DAG-CNNs. In: ICCV, pp. 1215–1223. IEEE Computer Society (2015)
Yu, W., Yang, K., Yao, H., Sun, X., Xu, P.: Exploiting the complementary strengths of multi-layer CNN features for image retrieval. Neurocomputing 237, 235–241 (2017)
Yu, Z., Dou, Z., Bakker, E.M., Wu, S.: Self-supervised asymmetric deep hashing with margin-scalable constraint for image retrieval (2021)
Zhang, L., Edraki, M., Qi, G.: CapProNet: deep feature learning via orthogonal projections onto capsule subspaces. In: NeurIPS, pp. 5819–5828 (2018)
Zhang, Z., Lan, C., Zeng, W., Jin, X., Chen, Z.: Relation-aware global attention for person re-identification. In: CVPR, pp. 3183–3192. IEEE (2020)
Zou, X., Wang, X., Bakker, E.M., Wu, S.: Multi-label semantics preserving based deep cross-modal hashing. Sig. Process. Image Commun. 93, 116131 (2021). https://doi.org/10.1016/j.image.2020.116131. https://www.sciencedirect.com/science/article/pii/S0923596520302344
Acknowledgements
This work was supported by the National Natural Science Foundation of China (61806168), Fundamental Research Funds for the Central Universities (SWU117059), and Venture and Innovation Support Program for Chongqing Overseas Returnees (CX2018075).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, X., Wu, S., Xiao, G. (2021). Global Fusion Capsule Network with Pairwise-Relation Attention Graph Routing. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13108. Springer, Cham. https://doi.org/10.1007/978-3-030-92185-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-92185-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92184-2
Online ISBN: 978-3-030-92185-9
eBook Packages: Computer ScienceComputer Science (R0)