Skip to main content
SpringerLink
Log in

Exploring semantic awareness via graph representation for text classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Text classification is a fundamental problem in natural language processing. Nowadays, text classification based on GNN attracts the attention of researchers. However, the existing works not fulfill well the transmission of contextual semantic information, and they pay more attention to capturing the local features instead of global. Such methods ignore the importance of keyword information features, so they can not fully mine the text-level semantic representation. To relieve such problems, we propose the GText model for discovering the basic features with words and establishing a deeper relationship representation between words and documents. Specially, we utilize semantic features graphs to achieve text semantic representation. Meanwhile, we propose semantic information passing(SIP) mechanism to transmit contextual semantic information, which can enhance the semantic representation from multi-views. In addition, the gate mechanism can further mine the explicit keywords of the whole document. With GText, the test accuracy on MR improved about 2% and on Ohsumed at most 9%, which illustrates GText can better achieve the mining and transmission of text semantic information. Experiments on several authoritative datasets show that our method is superior to the existing text classification methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Su J, Carreras X, Duh K (eds) Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016. The Association for Computational Linguistics, pp 606–615

  2. Wang SI, Manning CD (2012) Baselines and bigrams: Simple, good sentiment and topic classification. In: The 50th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, July 8-14, 2012, Jeju Island, Korea - Volume 2: Short Papers. The Association for Computer Linguistics, pp 90–94

  3. Lipton ZC, Kale DC, Elkan C, Wetzel RC (2016) Learning to diagnose with LSTM recurrent neural networks. In: Bengio Y, LeCun Y (eds) 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings

  4. Androutsopoulos I, Koutsias J, Chandrinos K, Paliouras G, Spyropoulos C D (2000) An evaluation of naive bayesian anti-spam filtering. arXiv:https://arxiv.org/abs/cs/0006013

  5. Tan S (2006) An effective refinement strategy for knn text classifier. Expert Syst Appl 30 (2):290–298

    Article  Google Scholar 

  6. Forman G (2008) Bns feature scaling: An improved representation over tf-idf for svm text classification. In: Proceedings of the 17th ACM Conference on Information and Knowledge Management, CIKM ’08. Association for Computing Machinery, New York, pp 263–270

  7. Kim Y (2014) Convolutional neural networks for sentence classification. In: Moschitti A, Pang B, Daelemans W (eds) Proceedings of the 2014 conference on empirical methods in natural language processing, EMNLP 2014, october 25-29, 2014, doha, qatar, A meeting of sigdat, a special interest group of the ACL. ACL, pp 1746–1751

  8. Mikolov T, Karafiát M, Burget L, Cernocký J, Khudanpur S (2010) Recurrent neural network based language model. In: Kobayashi T, Hirose K, Nakamura S (eds) INTERSPEECH 2010, 11th annual conference of the international speech communication association, makuhari, chiba, japan, september 26-30, 2010. ISCA, pp 1045–1048

  9. Liu P, Qiu X, Huang X (2016) Recurrent neural network for text classification with multi-task learning. In: Kambhampati S (ed) Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016. IJCAI/AAAI Press, pp 2873–2879

  10. Lai S, Xu L, Liu K, Zhao J (2015) Recurrent convolutional neural networks for text classification. In: Bonet B, Koenig S (eds) Proceedings of the twenty-ninth AAAI conference on artificial intelligence. AAAI Press, Austin, pp 2267–2273

  11. Joulin A, Grave E, Bojanowski P, Mikolov T (2017) Bag of tricks for efficient text classification. In: Lapata M, Blunsom P, Koller A (eds) Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Volume 2: Short Papers. Association for Computational Linguistics, Valencia, pp 427–431

  12. Peng H, Li J, He Y, Liu Y, Bao M, Wang L, Song Y, Yang Q (2018) Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In: Champin P-A, Gandon F, Lalmas M, Ipeirotis P G (eds) Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW 2018, address=Lyon. ACM, pp 1063–1072

  13. Kipf T N, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net

  14. Cai H, Zheng VW, Chang KC-C (2018) A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Trans Knowl Data Eng 30(9):1616–1637

    Article  Google Scholar 

  15. Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019. AAAI Press, Honolulu, pp 7370–7377

  16. Li Z, Cui Z, Wu S, Zhang X, Wang L (2019) Fi-gnn: Modeling feature interactions via graph neural networks for CTR prediction. In: Zhu W, Tao D, Cheng X, Cui P, Rundensteiner EA, Carmel D, He Q, Yu JX (eds) Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, address=Beijing. ACM, pp 539–548

  17. Zhang Y, Yu X, Cui Z, Wu S, Wen Z, Wang L (2020) Every document owns its structure: Inductive text classification via graph neural networks. In: Jurafsky D, Chai J, Schluter N, Tetreault JR (eds) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020. Association for Computational Linguistics, pp 334–339

  18. Liu X, You X, Zhang X, Wu J, Lv P (2020) Tensor graph convolutional networks for text classification. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020. AAAI Press, New York, pp 8409–8416

  19. Rousseau F, Kiagias E, Vazirgiannis M (2015) Text categorization as a graph classification problem. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, Volume 1: Long Papers. The Association for Computer Linguistics, Beijing, pp 1702–1712

  20. Li P, Zhong P, Mao K, Wang D, Yang X, Liu Y, Yin J, See S (2021) ACT: an attentive convolutional transformer for efficient text classification. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event. AAAI Press, pp 13261–13269

  21. Arous I, Dolamic L, Yang J, Bhardwaj A, Cuccu G, Cudré-Mauroux P (2021) MARTA: leveraging human rationales for explainable text classification. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event. AAAI Press, pp 5868–5876

  22. Du J, Huang Y, Moilanen K (2021) Knowledge-aware leap-lstm: Integrating prior knowledge into leap-lstm towards faster long text classification. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event. AAAI Press, pp 12768–12775

  23. Lee JH, Ko S-K, Han Y-S (2021) Salnet: Semi-supervised few-shot text classification with attention-based lexicon construction. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event. AAAI Press, pp 13189–13197

  24. Li X, Li Z, Xie H, Li Q (2021) Merging statistical feature via adaptive gate for improved text classification. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event. AAAI Press, pp 13288–13296

  25. Jiang T, Wang D, Sun L, Yang H, Zhao Z, Zhuang F (2021) Lightxml: Transformer with dynamic negative sampling for high-performance extreme multi-label text classification. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event. AAAI Press, pp 7987–7994

  26. Xiao L, Zhang X, Jing L, Huang C, Song M (2021) Does head label help for long-tailed multi-label text classification. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event. AAAI Press, pp 14103–14111

  27. Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi VF, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, Gülçehre C, Song HF, Ballard AJ, Gilmer J, Dahl GE, Vaswani A, Allen KR, Nash C, Langston V, Dyer C, Heess N, Wierstra D, Kohli P, Botvinick M, Vinyals O, Li Y, Pascanu R (2018) Relational inductive biases, deep learning, and graph networks. arXiv:1806.01261

  28. Wang G, Li C, Wang W, Zhang Y, Shen D, Zhang X, Henao R, Carin L (2018) Joint embedding of words and labels for text classification. In: Gurevych I, Miyao Y (eds) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Volume 1: Long Papers. Association for Computational Linguistics, Melbourne, pp 2321–2331

Download references

Acknowledgements

This work was supported in part by the National Social Science Foundation under Award 19BYY076, in part Key R & D project of Shandong Province 2019JZZY010129, and in part by the Shandong Provincial Social Science Planning Project under Award 19BJCJ51, Award 18CXWJ01, and Award 18BJYJ04.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, China

    Yahui Li, Yifan Liu, Zhenfang Zhu & Peiyu Liu

Authors
  1. Yahui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yifan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenfang Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peiyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peiyu Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Liu, Y., Zhu, Z. et al. Exploring semantic awareness via graph representation for text classification. Appl Intell 53, 2088–2097 (2023). https://doi.org/10.1007/s10489-022-03526-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03526-z

Keywords

Search

Navigation