Skip to main content

Advertisement

SpringerLink
Log in

Image classification based on quaternion-valued capsule network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, a novel quaternion-valued (QV) capsule module is designed to construct QV capsule networks for image classification. The quaternion algebra is introduced into the capsule networks to effectively capture the external dependencies and internal structural information. Moreover, the QV capsules can enhance the representation of complex information and alleviate the information loss of vanilla capsule networks. Particularly, a non-iterative quaternion routing algorithm is proposed to integrate QV capsules, considering both the membership and the consistency of QV capsules in two stages. Extensive experiments are conducted on classic image datasets, hyperspectral image datasets, and face datasets, which demonstrate that: firstly, the QV capsule network achieves higher classification accuracy, reaching 92.95% in UC Merced Land Use and 95.02% in CIFAR 10; secondly, the QV capsule module is more adaptable to different backbone networks than the vanilla capsule module; finally, the QV capsule network shows high performance with limited training samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Alam M, Samad MD, Vidyaratne L, Glandon A, Iftekharuddin KM (2020) Survey on deep neural networks in speech and vision systems. Neurocomputing 417:302–321. https://doi.org/10.1016/j.neucom.2020.07.053

    Article  Google Scholar 

  2. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2016.90, pp 770–778

  3. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2015.7298594, pp 1–9

  4. Hinton GE, Krizhevsky A, Wang SD (2011) Transforming auto-encoders. In: Artificial Neural Networks and Machine Learning – ICANN 2011. https://doi.org/10.1007/978-3-642-21735-7_6. Springer, pp 44–51

  5. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Proceedings of the 31st international conference on neural information processing systems, pp 3859–3869

  6. Hinton GE, Sabour S, Frosst N (2018) Matrix capsules with EM routing. In: 6th International conference on learning representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018. Conference Track Proceedings. https://openreview.net/forum?id=HJWLfGWRb, pp 1–15

  7. LaLonde R, Xu Z, Irmakci I, Jain S, Bagci U (2021) Capsules for biomedical image segmentation. Med Image Anal 68:89–101908. https://doi.org/10.1016/j.media.2020.101889

    Article  Google Scholar 

  8. Pérez E, Ventura S (2021) Melanoma recognition by fusing convolutional blocks and dynamic routing between capsules. Cancers 13(19):4974–4993. https://doi.org/10.3390/cancers13194974

    Article  Google Scholar 

  9. Parcollet T, Morchid M, Linarès G (2019) Quaternion convolutional neural networks for heterogeneous image processing. In: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP.2019.8682495, pp 8514–8518

  10. Jing B, Prabhu V, Gu A, Whaley J (2021) Rotation-invariant gait identification with quaternion convolutional neural networks (student abstract). In: Proceedings of the AAAI conference on artificial intelligence, vol 35. pp 15805–15806. https://ojs.aaai.org/index.php/AAAI/article/view/17899

  11. Grassucci E, Comminiello D, Uncini A (2021) A quaternion-valued variational autoencoder. In: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP39728.2021.9413859, pp 3310–3314

  12. Xiang M, Dees BS, Mandic DP (2018) Multiple-model adaptive estimation for 3-d and 4-d signals: A widely linear quaternion approach. IEEE Trans Neural Netw Learn Syst 30(1):72–84. https://doi.org/10.1109/TNNLS.2018.2829526

    Article  Google Scholar 

  13. Gu J, Tresp V, Hu H (2021) Capsule network is not more robust than convolutional network. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR46437.2021.01408, pp 14304–14312

  14. Byerly A, Kalganova T, Dear I (2021) No routing needed between capsules. Neurocomputing 463:545–553. https://doi.org/10.1016/j.neucom.2021.08.064

    Article  Google Scholar 

  15. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2017.243, pp 2261–2269

  16. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2017.195, pp 1800–1807

  17. Zhang T, Qi G, Xiao B, Wang J (2017) Interleaved group convolutions. In: IEEE International conference on computer vision, ICCV 2017, Venice, Italy, October 22-29, 2017. https://doi.org/10.1109/ICCV.2017.469, pp 4383–4392

  18. Zhang X, Zhou X, Lin M, Sun J (2018) Shufflenet: An extremely efficient convolutional neural network for mobile devices. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. https://doi.org/10.1109/CVPR.2018.00716, pp 6848–6856

  19. Ma N, Zhang X, Zheng H-T , Sun J (2018) Shufflenet v2: Practical guidelines for efficient cnn architecture design. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 116–131

  20. Kalyani G, Janakiramaiah B, Karuna A, Prasad L (2021) Diabetic retinopathy detection and classification using capsule networks. Complex Intell Syst. https://doi.org/10.1007/s40747-021-00318-9

  21. Dinani ST, Caragea D (2021) Disaster image classification using capsule networks. In: 2021 International Joint Conference on Neural Networks (IJCNN). https://doi.org/10.1109/IJCNN52387.2021.9534448, pp 1–8

  22. Hsu J-T, Kuo C-H, Chen D-W (2020) Image super-resolution using capsule neural networks. IEEE Access 8:9751–9759. https://doi.org/10.1109/ACCESS.2020.2964292

    Article  Google Scholar 

  23. Sun K, Yuan L, Xu H, Wen X (2020) Deep tensor capsule network. IEEE Access 8:96920–96933. https://doi.org/10.1109/ACCESS.2020.2996282

    Article  Google Scholar 

  24. Gu J, Tresp V (2020) Improving the robustness of capsule networks to image affine transformations. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR42600.2020.00731, pp 7283–7291

  25. Xiang C, Zhang L, Tang Y, Zou W, Xu C (2018) Ms-capsnet: A novel multi-scale capsule network. IEEE Signal Process Lett 25(12):1850–1854. https://doi.org/10.1109/LSP.2018.2873892

    Article  Google Scholar 

  26. Pucci R, Micheloni C, Foresti G L, Martinel N (2020) Deep interactive encoding with capsule networks for image classification. Multimed Tools Appl 79(43):32243–32258. https://doi.org/10.1007/s11042-020-09455-8

    Article  Google Scholar 

  27. Sun K, Wen X, Yuan L, Xu H (2021) Dense capsule networks with fewer parameters. Soft Comput 25(10):6927–6945. https://doi.org/10.1007/s00500-021-05774-6

    Article  Google Scholar 

  28. Sun G, Ding S, Sun T, Zhang C, Du W (2022) A novel dense capsule network based on dense capsule layers. Appl Intell 52(3):3066–3076. https://doi.org/10.1007/s10489-021-02630-w

    Article  Google Scholar 

  29. Amer M, Maul T (2020) Path capsule networks. Neural Process Lett 52(1):545–559. https://doi.org/10.1007/s00500-021-05774-6

    Article  Google Scholar 

  30. Huang W, Zhou F (2020) Da-capsnet: dual attention mechanism capsule network. Sci Rep 10(1):1–13. https://doi.org/10.1038/s41598-020-68453-w

    MathSciNet  Google Scholar 

  31. Peer D, Stabinger S, Rodríguez-Sánchez A (2021) Limitation of capsule networks. Pattern Recog Lett 144:68–74. https://doi.org/10.1016/j.patrec.2021.01.017

    Article  Google Scholar 

  32. Rajasegaran J, Jayasundara V, Jayasekara S, Jayasekara H, Seneviratne S, Rodrigo R (2019) Deepcaps: Going deeper with capsule networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2019.01098, pp 10717–10725

  33. Yang S, Lee F, Miao R, Cai J, Chen L, Yao W, Kotani K, Chen Q (2020) Rs-capsnet: An advanced capsule network. IEEE Access 8:85007–85018. https://doi.org/10.1109/ACCESS.2020.2992655

    Article  Google Scholar 

  34. Pucci R, Micheloni C, Martinel N (2021) Self-attention agreement among capsules. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). https://doi.org/10.1109/ICCVW54120.2021.00035, pp 272–280

  35. Mazzia V, Salvetti F, Chiaberge M (2021) Efficient-CapsNet: capsule network with self-attention routing. Sci Rep 11(1):14634–14647. https://doi.org/10.1038/s41598-021-93977-0

    Article  Google Scholar 

  36. Zhao Z, Cheng S (2021) Capsule networks with non-iterative cluster routing. Neural Netw 143:690–697. https://doi.org/10.1016/j.neunet.2021.07.032

    Article  Google Scholar 

  37. Li Y, Zhao W, Cambria E, Wang S, Eger S (2021) Graph routing between capsules. Neural Netw 143:345–354. https://doi.org/10.1016/j.neunet.2021.06.018

    Article  Google Scholar 

  38. Tygert M, Bruna J, Chintala S, LeCun Y, Piantino S, Szlam A (2016) A mathematical motivation for complex-valued convolutional networks. Neural Comput 28(5):815–825. https://doi.org/10.1162/neco_a_00824

    Article  MathSciNet  MATH  Google Scholar 

  39. Zhang H, Liu AQ (2021) An optical computing chip executing complex-valued neural network and its on-chip training. In: Katayama R, Takashima Y (eds) ODS 2021: industrial optical devices and systems. https://doi.org/10.1117/12.2597553. SPIE, pp 457–468

  40. Xu F, Zhang J, Fang T, Huang S, Wang M (2018) Synchronous dynamics in neural system coupled with memristive synapse. Nonlinear Dyn 92(3):1395–1402. https://doi.org/10.1007/s11071-018-4134-0

    Article  Google Scholar 

  41. Protachevicz PR, Borges RR, Reis AS, Borges FS, Iarosz KC, Caldas IL, Lameu EL, Macau EEN, Viana RL, Sokolov IM, Ferrari FAS, Kurths J, Batista AM, Lo C-Y, He Y, Lin C-P (2018) Synchronous behaviour in network model based on human cortico-cortical connections. Physiol Meas 39(7):074006. https://doi.org/10.1088/1361-6579/aace91

    Article  Google Scholar 

  42. Guo Y, Gao Z, Liu Y, Li S, Zhu J, Chen P, Liu B-F (2020) Multichannel synchronous hydrodynamic gating coupling with concentration gradient generator for high-throughput probing dynamic signaling of single cells. Anal Chem 92(17):12062–12070. https://doi.org/10.1021/acs.analchem.0c02746

    Article  Google Scholar 

  43. Yin Q, Wang J, Luo X, Zhai J, Jha SK, Shi Y-Q (2019) Quaternion convolutional neural network for color image classification and forensics. IEEE Access 7:20293–20301. https://doi.org/10.1109/ACCESS.2019.2897000

    Article  Google Scholar 

  44. Parcollet T, Morchid M, Linarès G (2020) A survey of quaternion neural networks. Artif Intell Rev 53(4):2957–2982. https://doi.org/10.1007/s10462-019-09752-1

    Article  Google Scholar 

  45. Popa C-A (2018) Learning algorithms for quaternion-valued neural networks. Neural Process Lett 47(3):949–973. https://doi.org/10.1007/s11063-017-9716-1

    Article  Google Scholar 

  46. Zhang A, Tay Y, Zhang S, Chan A, Luu AT, Hui SC, Fu J (2021) Beyond fully-connected layers with quaternions: Parameterization of hypercomplex multiplications with 1/n parameters 9Th international conference on learning representations, ICLR, pp 1–13

  47. Kosiorek A, Sabour S, Teh YW, Hinton GE (2019) Stacked capsule autoencoders. In: Advances in Neural Information Processing Systems, vol 32. https://proceedings.neurips.cc/paper/2019/file/2e0d41e02c5be4668ec1b0730b3346a8-Paper.pdf, pp 1–11

  48. Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems - GIS’10. https://doi.org/10.1145/1869790.1869829, pp 270–279

  49. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications Preprint at arXiv:1704.04861

  50. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition. https://doi.org/10.1109/CVPR.2018.00474, pp 4510–4520

  51. Huang K-K, Ren C-X, Liu H, Lai Z-R, Yu Y-F, Dai D-Q (2021) Hyperspectral image classification via discriminative convolutional neural network with an improved triplet loss. Pattern Recog 112:107744–107757. https://doi.org/10.1016/j.patcog.2020.107744

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Nos. 11871104 and 12131006). The authors would like to express their gratitude to the reviewers for their insightful remarks and ideas on how to improve the paper’s quality.

Author information

Authors and Affiliations

  1. College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China

    Heng Zhou

  2. Beijing Zhongdi Runde Petroleum Technology Co., Ltd., Haidian District, Beijing, 100083, Beijing, China

    Chunlei Zhang

  3. School of Statistics, Beijing Normal University, Haidian District, Beijing, 100875, Beijing, China

    Xin Zhang

  4. School of Science, China University of Geosciences, Haidian District, Beijing, 100083, Beijing, China

    Qiaoyu Ma

Authors
  1. Heng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunlei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiaoyu Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunlei Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, H., Zhang, C., Zhang, X. et al. Image classification based on quaternion-valued capsule network. Appl Intell 53, 5587–5606 (2023). https://doi.org/10.1007/s10489-022-03849-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03849-x

Keywords

Search

Navigation