Skip to main content
SpringerLink
Log in

CharCaps: Character-Level Text Classification Using Capsule Networks

  • Conference paper
  • First Online:
Advanced Intelligent Computing Technology and Applications (ICIC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14087))

Included in the following conference series:

  • 979 Accesses

Abstract

Text classification is a hot topic in the field of natural language processing and has achieved great success. Existing character-level text classification methods mainly use convolutional neural networks to extract character-level local features, making them ineffective in modeling the hierarchical spatial relationship information on the character-level features, reducing the classification performance. This paper proposes a new character-level text classification framework based on the capsule network called CharCaps to solve the above problem. The proposed CharCaps framework first extracts character-level text features using seven convolutional layers and then reconstructs them based on the capsule vector representation to obtain the hierarchical spatial relationship information between character-level features effectively and achieve a significant classification without pre-trained models. Experimental results on five challenging benchmark datasets demonstrate that our proposed method outperforms state-of-the-art character-level text classification models, especially convolutional neural network-based models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wan, J., Li, J., Lai, Z., Du, B., Zhang, L.: Robust face alignment by cascaded regression and de-occlusion. Neural Netw. 123, 261–272 (2020)

    Article  Google Scholar 

  2. Wan, J., et al.: Robust facial landmark detection by cross-order cross-semantic deep network. Neural Netw. 136, 233–243 (2021)

    Article  Google Scholar 

  3. Wu, Y., Li, J., Song, C., Chang, J.: Words in pairs neural networks for text classiffcation. Chin. J. Electron. 29, 491–500 (2020)

    Article  Google Scholar 

  4. Sergio, G.C., Lee, M.: Stacked debert: all attention in incomplete data for text classiffcation. Neural Netw. 136, 87–96 (2021)

    Article  Google Scholar 

  5. Kim, Y.: Convolutional Neural Networks for Sentence Classification. In: Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751. ACL, Doha, Qatar (2014)

    Google Scholar 

  6. Liu, P., Qiu, X., Huang, X.: Recurrent Neural Network for Text Classification with Multi-Task Learning. In: 25th International Joint Conference on Artificial Intelligence, pp. 2873–2879. IJCAI/AAAI Press, New York, NY, USA (2016)

    Google Scholar 

  7. Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent Convolutional Neural Networks for Text Classification. In: 29th AAAI Conference on Artificial Intelligence, pp. 2267–2273. AAAI Press, Austin, Texas, USA (2015)

    Google Scholar 

  8. Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. In: 31th International Conference on Machine Learning, pp. 1188–1196. JMLR, Beijing, China (2014)

    Google Scholar 

  9. Pennington, J., Socher, R., Manning, C.D.: Contribution title. In: Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543. ACL, Doha, Qatar (2014)

    Google Scholar 

  10. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186. ACL, Minneapolis, MN, USA (2019)

    Google Scholar 

  11. Mekala, D., Shang, J.: Contextualized Weak Supervision for Text Classification. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 323–333. ACL, Online (2020)

    Google Scholar 

  12. Croce, D., Castellucci, G., Basili, R.: GAN-BERT: generative adversarial learning for robust text classification with a bunch of labeled examples. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 2114–2119. ACL, Online (2020)

    Google Scholar 

  13. Qin, Q., Hu, W., Liu, B.: Feature projection for improved text classification. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 8161–8171. ACL, Online (2020)

    Google Scholar 

  14. Chen, H., Zheng, G., Ji, Y.: Generating hierarchical explanations on text classification via feature interaction detection. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 5578–5593. ACL, Online (2020)

    Google Scholar 

  15. Zhang, X., Zhao, J., LeCun, Y.: Character-level Convolutional Networks for Text Classification. In: 28th Annual Conference on Neural Information Processing Systems, pp. 649–657. Montreal, Quebec, Canada (2015)

    Google Scholar 

  16. Kim, Y., Jernite, Y., Sontag, D., Rush, A.M.: Character-aware neural language models. In: 9th International Proceedings on Proceedings, pp. 2741–2749. AAAI Press, Phoenix, Arizona, USA (2016)

    Google Scholar 

  17. Liu, B., Zhou, Y., Sun, W.: Character-level text classification via convolutional neural network and gated recurrent unit. Int. J. Mach. Learn. Cybern. 11(8), 1939–1949 (2020). https://doi.org/10.1007/s13042-020-01084-9

    Article  Google Scholar 

  18. Londt, T., Gao, X., Andreae, P.: Evolving character-level densenet architectures using genetic programming. In: Castillo, P.A., JiménezLaredo, J.L. (eds.) Applications of Evolutionary Computation. LNCS, vol. 12694, pp. 665–680. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72699-7_42

    Chapter  Google Scholar 

  19. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: 30th Annual Conference on Neural Information Processing, pp. 3856–3866. Long Beach, CA, USA (2017)

    Google Scholar 

  20. Wu, Y., Li, J., Chen, V., Chang, J., Ding, Z., Wang, Z.: Text classification using triplet capsule networks. in: international joint conference on neural networks, pp. 1–7. IEEE, Glasgow, United Kingdom (2020)

    Google Scholar 

  21. Wu, Y., Li, J., Wu, J., Chang, J.: Siamese capsule networks with global and local features for text classification. Neurocomputing 390, 88–98 (2020)

    Article  Google Scholar 

  22. Hong, S.K., Jang, T.: LEA: meta knowledge-driven self-attentive document embedding for few-shot text classification. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 99–106. ACL, Seattle, WA, United States (2022)

    Google Scholar 

  23. Wang, J., et al.: Towards Unified Prompt Tuning for Few-shot Text Classification. In: Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 524–536. Publisher, Abu Dhabi, United Arab Emirates (2022)

    Google Scholar 

  24. Shnarch, E., et al.: Cluster & tune: boost cold start performance in text classification. In: 60th Annual Meeting of the Association for Computational Linguistics, pp. 7639–7653. ACL, Dublin, Ireland (2022)

    Google Scholar 

  25. Tsai, Y.H., Srivastava, N., Goh, H., Salakhutdinov, R.: Capsules with inverted dot-product attention routing. In: 8th International Conference on Learning Representations, ICLR, Addis Ababa, Ethiopia (2020)

    Google Scholar 

  26. Gong, J., Qiu, X., Wang, S., Huang, X.: Information aggregation via dynamic routing for sequence encoding. In: 27th International Conference on Computational Linguistics, pp. 2742–2752. COLING, Santa Fe, New Mexico, USA (2018)

    Google Scholar 

  27. Wang, Y., Sun, A., Han, J., Liu, Y., Zhu, X.: Sentiment analysis by capsules. In: Conference on World Wide Web, pp. 1165–1174. ACM, Lyon, France (2018)

    Google Scholar 

  28. Yang, M., Zhao, W., Chen, L., Qu, Q., Zhao, Z., Shen, Y.: Investigating the transferring capability of capsule networks for text classification. Neural Netw. 118, 247–261 (2019)

    Article  Google Scholar 

  29. Zhao, W., Peng, H., Eger, S., Cambria, E., Yang, M.: Towards scalable and reliable capsule networks for challenging NLP applications. In: 57th Conference of the Association for Computational Linguistic, pp. 1549–1559. ACL, Florence, Italy (2019)

    Google Scholar 

  30. Chen, Z., Qian, T.: Transfer capsule network for aspect level sentiment classification. In: 57th Conference of the Association for Computational Linguistic, pp. 547–556. ACL, Florence, Italy (2019)

    Google Scholar 

  31. Lehmann, J., et al.: DBpedia - a large-scale, multilingual knowledge base extracted from Wikipedia. Semant. Web 6(2), 167–195 (2015)

    Article  Google Scholar 

  32. McAuley, J., Leskovec, J.: Hidden factors and hidden topics: understanding rating dimensions with review text. In: 7th ACM Conference on Recommender Systems, pp. 165–172. ACM, Hong Kong, China (2013)

    Google Scholar 

  33. Rojas, K.R., Bustamante, G., Cabezudo, M.A.S., Oncevay, A.: Efficient Strategies for Hierarchical Text Classification: External Knowledge and Auxiliary Tasks. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 2252–2257. ACL, Online (2020)

    Google Scholar 

  34. Chen, J., Yang, Z., Yang, D.: MixText: linguistically-informed interpolation of hidden space for semi-supervised text classification. In: 58th Annual Meeting of the Association for Computational Linguistics, pp. 2147–2157. ACL, Online (2020)

    Google Scholar 

  35. Sinha, K., Dong, Y., Cheung, J.C.K., Ruths, D.: A hierarchical neural attention-based text classifier. In: Conference on Empirical Methods in Natural Language Processing, pp. 817–823. ACL, Brussels, Belgium (2018)

    Google Scholar 

Download references

Acknowledgments

This work was Sponsored by Natural Science Foundation of Shanghai (No. 22ZR1445000) and Research Foundation of Shanghai Sanda University (No. 2020BSZX005, No. 2021BSZX006).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Sanda University, Shanghai, 201209, China

    Yujia Wu, Xin Guo & Kangning Zhan

Authors
  1. Yujia Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kangning Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yujia Wu or Xin Guo .

Editor information

Editors and Affiliations

  1. Department of Computer Science, Eastern Institute of Technology, Zhejiang, China

    De-Shuang Huang

  2. University of Wollongong, North Wollongong, NSW, Australia

    Prashan Premaratne

  3. Zhengzhou University of Light Industry, Zhengzhou, China

    Baohua Jin

  4. Zhong Yuan University of Technology, Zhengzhou, China

    Boyang Qu

  5. University of Ulsan, Ulsan, Korea (Republic of)

    Kang-Hyun Jo

  6. Department of Computer Science, Liverpool John Moores University, Liverpool, UK

    Abir Hussain

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wu, Y., Guo, X., Zhan, K. (2023). CharCaps: Character-Level Text Classification Using Capsule Networks. In: Huang, DS., Premaratne, P., Jin, B., Qu, B., Jo, KH., Hussain, A. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2023. Lecture Notes in Computer Science, vol 14087. Springer, Singapore. https://doi.org/10.1007/978-981-99-4742-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-4742-3_15

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-4741-6

  • Online ISBN: 978-981-99-4742-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation