Abstract
Deep Learning (DL) has been one of the preferred techniques for Natural Language Processing (NLP) applications. Due to its nature, a text can be better represented in a graph structure, when compared with the classical feature-based representations. Therefore, several researchers have explored the use of Graph Neural Networks (GNN) for text analysis. GNNs show excellent results in text classification tasks, given their property of capturing contextual and global information in a corpus. The Text Graph Convolutional Network (TGCN) showed the ability to outperform traditional NLP methods in benchmark classification tasks. However, this method has a very high memory cost for the text graph construction. By exploring the results of text representations, we propose a new method to generate a text graph, capable of influencing the result of the TGCN, leading to a reduced use of memory.
Supported by FAPESP, CNPq and MackPesquisa.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., Gao, J.: Deep learning–based text classification: a comprehensive review. ACM Comput. Surv. (CSUR) 54(3) (2021)
Aggarwal, C.C., Zhai, C.X.: A survey of text classification algorithms. In: Mining Text Data, pp. 163–222. Springer, Boston (2012)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Proc. 45(11) (1997)
Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751 (2014)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks (2016). arXiv:1609.02907
Yang, J.X., Bai, L., Guo, Y.: A survey of text classification models. In: Proceedings of the 2020 2nd International Conference on Robotics, Intelligent Control and Artificial Intelligence, pp. 327–334 (2020)
Rousseau, F., Kiagias, E., Vazirgiannis, M.: Text categorization as a graph classification problem. In: ACL, vol. 15, p. 107 (2015)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks (2017). arXiv:1710.10903
Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 7370–7377 (2019)
Luo, Y., Uzuner, Ö., Szolovits, P.: Bridging semantics and syntax with graph algorithms-state-of-the-art of extracting biomedical relations. Brief. Bioinf. 18(1) (2017)
Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 2873–2879 (2016)
Bird, S.: Multidisciplinary instruction with the natural language toolkit. Association for Computational Linguistics (2008)
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017)
Wu, L., Chen, Y., Shen, K., Guo, X., Gao, H., Li, S., Pei, J., Long, B.: Graph neural networks for natural language processing: a survey (2021). arXiv:2106.06090
Role, F., Nadif, M.: Handling the impact of low frequency events on co-occurrence based measures of word similarity. In: Proceedings of the International Conference on Knowledge Discovery and Information Retrieval (KDIR-2011). Scitepress, pp. 218–223 (2011)
Roelleke, T., Wang, J.: Tf-idf uncovered: a study of theories and probabilities. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 435–442 (2008)
Rousseau, F., Vazirgiannis, M.: Graph-of-word and TW-IDF: new approach to ad hoc IR. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 59–68 (2013)
Osman, A.H., Barukub, O.M.: Graph-based text representation and matching: a review of the state of the art and future challenges. IEEE Access 8 (2020)
Jin, W., Srihari, R.K.: Graph-based text representation and knowledge discovery. In: Proceedings of the 2007 ACM Symposium on Applied Computing, pp. 807–811 (2007)
Wang, Y., Ni, X., Sun, J.-T., Tong, Y., Chen, Z.: Representing document as dependency graph for document clustering. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pp. 2177–2180 (2011)
Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., McClosky, D.: The stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55–60 (2014)
Pennebaker, J.W., Francis, M.E., Booth, R.J.: Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates, 71 (2001)
Github https://github.com/vitormeriat/nlp-based-text-gcn. Accessed 6 June 2022
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pereira, V.C.M., de Castro, L.N. (2023). Natural Language Processing Based on a Text Graph Convolutional Network. In: Omatu, S., Mehmood, R., Sitek, P., Cicerone, S., Rodríguez, S. (eds) Distributed Computing and Artificial Intelligence, 19th International Conference. DCAI 2022. Lecture Notes in Networks and Systems, vol 583. Springer, Cham. https://doi.org/10.1007/978-3-031-20859-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-20859-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20858-4
Online ISBN: 978-3-031-20859-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)