Skip to main content
SpringerLink
Log in

Exploring Self-supervised Capsule Networks for Improved Classification with Data Scarcity

  • Conference paper
  • First Online:
Third International Conference on Image Processing and Capsule Networks (ICIPCN 2022)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 514))

Included in the following conference series:

Abstract

While Capsule Networks overcome many shortcomings of Convolutional Neural Networks by taking spatial hierarchies between features into consideration and providing a new approach to routing between layers, they are sensible towards overfitting. Self-supervised training improves the semantic understanding of a network significantly and improves generalization without requiring additional images. Therefore, self-supervised Capsule Networks are a promising approach to improve learning under data scarcity, which makes the combination interesting for a huge variety of applications. Our approach improves test accuracy by up to 11.7% for small data sets and by up to 11.5% for small and imbalanced data sets. Furthermore, we explore the synergies and characteristics of this innovative method combination and give a general overview on the possibilities of self-supervised Capsule Networks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Afshar, P., Heidarian, S., Naderkhani, F., Oikonomou, A., Plataniotis, K.N., Mohammadi, A.: COVID-caps: a capsule network-based framework for identification of COVID-19 cases from X-ray images. Pattern Recogn. Lett. 138, 638–643 (2020)

    Article  Google Scholar 

  2. Afshar, P., Naderkhani, F., Oikonomou, A., Rafiee, M.J., Mohammadi, A., Plataniotis, K.N.: MIXCAPS: a capsule network-based mixture of experts for lung nodule malignancy prediction. Pattern Recogn. 116, 107942 (2021)

    Article  Google Scholar 

  3. Albuquerque, I., Naik, N., Li, J., Keskar, N., Socher, R.: Improving out-of-distribution generalization via multi-task self-supervised pretraining. arXiv preprint arXiv:2003.13525 (2020)

  4. Andonie, R.: Extreme data mining: inference from small datasets. Int. J. Comput. Commun. Control 5(3), 280–291 (2010)

    Article  Google Scholar 

  5. Basak, S.C., Grunwald, G.D., Gute, B.D., Balasubramanian, K., Opitz, D.: Use of statistical and neural net approaches in predicting toxicity of chemicals. J. Chem. Inf. Comput. Sci. 40(4), 885–890 (2000)

    Article  Google Scholar 

  6. Bear, D.M., et al.: Learning physical graph representations from visual scenes. arXiv preprint arXiv:2006.12373 (2020)

  7. Chauhan, A., Babu, M., Kandru, N., Lokegaonkar, S.: Empirical study on convergence of capsule networks with various hyperparameters (2018)

    Google Scholar 

  8. Ding, X., Wang, N., Gao, X., Li, J., Wang, X.: Group reconstruction and max-pooling residual capsule network. In: IJCAI, pp. 2237–2243 (2019)

    Google Scholar 

  9. Dombetzki, L.A.: An overview over capsule networks. Network architectures and services (2018)

    Google Scholar 

  10. Du, B., Gao, X., Hu, W., Li, X.: Self-contrastive learning with hard negative sampling for self-supervised point cloud learning. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3133–3142 (2021)

    Google Scholar 

  11. Duan, Q., Lee, J.: Fast-developing machine learning support complex system research in environmental chemistry. New J. Chem. 44(4), 1179–1184 (2020)

    Article  Google Scholar 

  12. Edstedt, J.: Towards Understanding Capsule Networks (2020)

    Google Scholar 

  13. Fang, C., Shang, Y., Xu, D.: Improving protein gamma-turn inception capsule networks prediction using inception capsule networks. Sci. Rep. 8, 15741 (2018)

    Article  Google Scholar 

  14. Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., Cord, M.: Boosting few-shot visual learning with self-supervision. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8059–8068 (2019)

    Google Scholar 

  15. Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728 (2018)

  16. Hendrycks, D., Mazeika, M., Kadavath, S., Song, D.: Using self-supervised learning can improve model robustness and uncertainty. arXiv preprint arXiv:1906.12340 (2019)

  17. Hinton, G.E., Krizhevsky, A., Wang, S.D.: Transforming auto-encoders. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011. LNCS, vol. 6791, pp. 44–51. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21735-7_6

    Chapter  Google Scholar 

  18. Hu, Y., et al.: An improved algorithm for imbalanced data and small sample size classification. J. Data Anal. Inf. Process. 3(03), 27 (2015)

    Google Scholar 

  19. Huang, R., Li, J., Wang, S., Li, G., Li, W.: A robust weight-shared capsule network for intelligent machinery fault diagnosis. IEEE Trans. Industr. Inf. 16(10), 6466–6475 (2020)

    Article  Google Scholar 

  20. Hutchinson, M.L., Antono, E., Gibbons, B.M., Paradiso, S., Ling, J., Meredig, B.: Overcoming data scarcity with transfer learning. arXiv preprint arXiv:1711.05099 (2017)

  21. Jayaraman, D., Grauman, K.: Learning image representations tied to ego-motion. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1413–1421 (2015)

    Google Scholar 

  22. de Jesus, D.R., Cuevas, J., Rivera, W., Crivelli, S.: Capsule networks for protein structure classification and prediction. arXiv preprint arXiv:1808.07475 (2018)

  23. Jia, B., Huang, Q.: DE-CapsNet: a diverse enhanced capsule network with disperse dynamic routing. Appl. Sci. 10(3), 884 (2020)

    Article  Google Scholar 

  24. Jiménez-Sánchez, A., Albarqouni, S., Mateus, D.: Capsule networks against medical imaging data challenges. In: Stoyanov, D., et al. (eds.) LABELS/CVII/STENT -2018. LNCS, vol. 11043, pp. 150–160. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01364-6_17

    Chapter  Google Scholar 

  25. Jing, L., Tian, Y.: Self-supervised visual feature learning with deep neural networks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43, 4037–4058 (2020)

    Article  Google Scholar 

  26. Kim, J., Jang, S., Park, E., Choi, S.: Text classification using capsules. Neurocomputing 376, 214–221 (2020)

    Article  Google Scholar 

  27. Kim, M., Chi, S.: Detection of centerline crossing in abnormal driving using CapsNet. J. Supercomput. 75(1), 189–196 (2019)

    Article  Google Scholar 

  28. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  29. Kitchin, R., Lauriault, T.P.: Small data in the era of big data. GeoJournal 80(4), 463–475 (2014). https://doi.org/10.1007/s10708-014-9601-7

    Article  Google Scholar 

  30. Kolesnikov, A., Zhai, X., Beyer, L.: Revisiting self-supervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1920–1929 (2019)

    Google Scholar 

  31. Kruthika, K., Maheshappa, H., Initiative, A.D.N., et al.: CBIR system using capsule networks and 3D CNN for Alzheimer’s disease diagnosis. Inform. Med. Unlocked 14, 59–68 (2019)

    Article  Google Scholar 

  32. Kumar, A.D.: Novel deep learning model for traffic sign detection using capsule networks. arXiv preprint arXiv:1805.04424 (2018)

  33. LaLonde, R., Bagci, U.: Capsules for object segmentation. arXiv preprint arXiv:1804.04241 (2018)

  34. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  35. Lee, H., Hwang, S.J., Shin, J.: Rethinking data augmentation: self-supervision and self-distillation. arXiv preprint arXiv:1910.05872 (2019)

  36. Lee, J.D., Lei, Q., Saunshi, N., Zhuo, J.: Predicting what you already know helps: provable self-supervised learning. arXiv preprint arXiv:2008.01064 (2020)

  37. Li, B., Li, Y., Eliceiri, K.W.: Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328 (2021)

    Google Scholar 

  38. Li, H., Guo, X., Dai, B., Ouyang, W., Wang, X.: Neural network encapsulation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 266–282. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01252-6_16

    Chapter  Google Scholar 

  39. Lin, A., Li, J., Ma, Z.: On learning and learned representation with dynamic routing in capsule networks. arXiv preprint arXiv:1810.040412(7) (2018)

  40. Liu, L., Liu, S.S., Yu, M., Zhang, J., Chen, F.: Concentration addition prediction for a multiple-component mixture containing no effect chemicals. Anal. Meth. 7(23), 9912–9917 (2015)

    Article  Google Scholar 

  41. Lu, C., Duan, S., Wang, L.: An improved capsule network based on newly reconstructed network and the method of sharing parameters. In: Lu, H., Tang, H., Wang, Z. (eds.) ISNN 2019. LNCS, vol. 11554, pp. 116–123. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22796-8_13

    Chapter  Google Scholar 

  42. Mahendran, A., Thewlis, J., Vedaldi, A.: Self-supervised segmentation by grouping optical-flow. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 528–534. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11021-5_31

    Chapter  Google Scholar 

  43. Mandal, B., Dubey, S., Ghosh, S., Sarkhel, R., Das, N.: Handwritten indic character recognition using capsule networks. In: 2018 IEEE Applied Signal Processing Conference (ASPCON), pp. 304–308. IEEE (2018)

    Google Scholar 

  44. Marchisio, A., Nanfa, G., Khalid, F., Hanif, M.A., Martina, M., Shafique, M.: CapsAttacks: robust and imperceptible adversarial attacks on capsule networks. arXiv preprint arXiv:1901.09878 (2019)

  45. Marcos, D., Volpi, M., Tuia, D.: Learning rotation invariant convolutional filters for texture classification. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2012–2017. IEEE (2016)

    Google Scholar 

  46. Nair, P., Doshi, R., Keselj, S.: Pushing the limits of capsule networks. arXiv preprint arXiv:2103.08074 (2021)

  47. Noroozi, M., Vinjimoor, A., Favaro, P., Pirsiavash, H.: Boosting self-supervised learning via knowledge transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9359–9367 (2018)

    Google Scholar 

  48. Ohri, K., Kumar, M.: Review on self-supervised image recognition using deep neural networks. Knowl. Based Syst. 224, 107090 (2021)

    Article  Google Scholar 

  49. Ohta, N., Kawai, S., Nobuhara, H.: Analysis and learning of capsule networks robust for small image deformation. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2020)

    Google Scholar 

  50. Ozbulak, G.: Image colorization by capsule networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 2150–2158 (2019)

    Google Scholar 

  51. Pasupa, K., Sunhem, W.: A comparison between shallow and deep architecture classifiers on small dataset. In: 2016 8th International Conference on Information Technology and Electrical Engineering (ICITEE), pp. 1–6. IEEE (2016)

    Google Scholar 

  52. Patrick, M.K., Adekoya, A.F., Mighty, A.A., Edward, B.Y.: Capsule networks-a survey. J. King Saud Univ. Comput. Inform. Sci. 34, 1295–1310 (2019)

    Google Scholar 

  53. Peer, D., Stabinger, S., Rodriguez-Sanchez, A.: Limitation of capsule networks. Pattern Recogn. Lett. 144, 68–74 (2021)

    Article  Google Scholar 

  54. Pinto, L., Gupta, A.: Supersizing self-supervision: learning to grasp from 50k tries and 700 robot hours. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 3406–3413. IEEE (2016)

    Google Scholar 

  55. Raghu, M., Zhang, C., Kleinberg, J., Bengio, S.: Transfusion: understanding transfer learning for medical imaging. arXiv preprint arXiv:1902.07208 (2019)

  56. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. arXiv preprint arXiv:1710.09829 (2017)

  57. Sabour, S., Tagliasacchi, A., Yazdani, S., Hinton, G.E., Fleet, D.J.: Unsupervised part representation by flow capsules. arXiv preprint arXiv:2011.13920 (2020)

  58. Sayed, N., Brattoli, B., Ommer, B.: Cross and learn: cross-modal self-supervision. In: Brox, T., Bruhn, A., Fritz, M. (eds.) GCPR 2018. LNCS, vol. 11269, pp. 228–243. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-12939-2_17

    Chapter  Google Scholar 

  59. Shahroudnejad, A., Afshar, P., Plataniotis, K.N., Mohammadi, A.: Improved explainability of capsule networks: relevance path by agreement. In: 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 549–553. IEEE (2018)

    Google Scholar 

  60. Sun, K., Yuan, L., Xu, H., Wen, X.: Deep tensor capsule network. IEEE Access 8, 96920–96933 (2020)

    Article  Google Scholar 

  61. Sun, W., et al.: Canonical capsules: self-supervised capsules in canonical pose. In: 35th Conference on Neural Information Processing Systems (2021)

    Google Scholar 

  62. Vijayakumar, T.: Comparative study of capsule neural network in various applications. J. Artif. Intell. 1(01), 19–27 (2019)

    Article  Google Scholar 

  63. Wang, D., Liang, Y., Xu, D.: Capsule network for protein post-translational modification site prediction. Bioinformatics 35(14), 2386–2394 (2019)

    Article  Google Scholar 

  64. Wang, Z., et al.: A novel method for intelligent fault diagnosis of bearing based on capsule neural network. Complexity 2019, 1–17 (2019)

    Google Scholar 

  65. Wu, F., Smith, J.S., Lu, W., Pang, C., Zhang, B.: Attentive prototype few-shot learning with capsule network-based embedding. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12373, pp. 237–253. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58604-1_15

    Chapter  Google Scholar 

  66. Xi, E., Bing, S., Jin, Y.: Capsule network performance on complex data. arXiv preprint arXiv:1712.03480 (2017)

  67. Xiang, C., Zhang, L., Tang, Y., Zou, W., Xu, C.: MS-CapsNet: a novel multi-scale capsule network. IEEE Signal Process. Lett. 25(12), 1850–1854 (2018)

    Article  Google Scholar 

  68. Xinyi, Z., Chen, L.: Capsule graph neural network. In: International Conference on Learning Representations (2018)

    Google Scholar 

  69. Xu, Q., Chen, K., Zhou, G., Sun, X.: Change capsule network for optical remote sensing image change detection. Remote Sens. 13(14), 2646 (2021)

    Article  Google Scholar 

  70. Yang, M., Zhao, W., Ye, J., Lei, Z., Zhao, Z., Zhang, S.: Investigating capsule networks with dynamic routing for text classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3110–3119 (2018)

    Google Scholar 

  71. Yang, S., et al.: RS-CapsNet: an advanced capsule network. IEEE Access 8, 85007–85018 (2020)

    Article  Google Scholar 

  72. Yang, Y., Xu, Z.: Rethinking the value of labels for improving class-imbalanced learning. arXiv preprint arXiv:2006.07529 (2020)

  73. Yuan, D., Chang, X., Huang, P.Y., Liu, Q., He, Z.: Self-supervised deep correlation tracking. IEEE Trans. Image Process. 30, 976–985 (2020)

    Article  Google Scholar 

  74. Zhang, N., Deng, S., Sun, Z., Chen, X., Zhang, W., Chen, H.: Attention-based capsule networks with dynamic routing for relation extraction. arXiv preprint arXiv:1812.11321 (2018)

  75. Zhang, T., et al.: Intelligent fault diagnosis of machines with small & imbalanced data: a state-of-the-art review and possible extensions. ISA Trans. 119, 152–171 (2022)

    Article  Google Scholar 

  76. Zhang, X., Luo, P., Hu, X., Wang, J., Zhou, J.: Research on classification performance of small-scale dataset based on capsule network. In: Proceedings of the 2018 4th International Conference on Robotics and Artificial Intelligence, pp. 24–28 (2018)

    Google Scholar 

  77. Zhao, T., Liu, Y., Huo, G., Zhu, X.: A deep learning IRIS recognition method based on capsule network architecture. IEEE Access 7, 49691–49701 (2019)

    Article  Google Scholar 

  78. Zhao, W., Ye, J., Yang, M., Lei, Z., Zhang, S., Zhao, Z.: Investigating capsule networks with dynamic routing for text classification. arXiv preprint arXiv:1804.00538 (2018)

  79. Zhao, Y., Birdal, T., Deng, H., Tombari, F.: 3D point capsule networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1009–1018 (2019)

    Google Scholar 

  80. Zhu, Z., Peng, G., Chen, Y., Gao, H.: A convolutional neural network based on a capsule network with strong generalization for bearing fault diagnosis. Neurocomputing 323, 62–75 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Friedrich-Schiller-University, Carl-Zeiß-Str. 3, 07743, Jena, Germany

    Ladyna Wittscher & Christian Pigorsch

Authors
  1. Ladyna Wittscher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Pigorsch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ladyna Wittscher .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, Da-Yeh University, Changhua, Taiwan

    Joy Iong-Zong Chen

  2. Departamento de Engenharia Mecânica, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal

    João Manuel R. S. Tavares

  3. Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA

    Fuqian Shi

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wittscher, L., Pigorsch, C. (2022). Exploring Self-supervised Capsule Networks for Improved Classification with Data Scarcity. In: Chen, J.IZ., Tavares, J.M.R.S., Shi, F. (eds) Third International Conference on Image Processing and Capsule Networks. ICIPCN 2022. Lecture Notes in Networks and Systems, vol 514. Springer, Cham. https://doi.org/10.1007/978-3-031-12413-6_4

Download citation

Publish with us

Policies and ethics

Search

Navigation