Skip to main content
SpringerLink
Log in

Label-representative graph convolutional network for multi-label text classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-label text classification (MLTC) is the task that assigns each document to the most relevant subset of class labels. Previous works usually ignored the correlation and semantics of labels resulting in information loss. To deal with this problem, we propose a new model that explores label dependencies and semantics by using graph convolutional networks (GCN). Particularly, we introduce an efficient correlation matrix to model label correlation based on occurrence and co-occurrence probabilities. To enrich the semantic information of labels, we design a method to use external information from Wikipedia for label embeddings. Correlated label information learned from GCN is combined with fine-grained document representation generated from another sub-net for classification. Experimental results on three benchmark datasets show that our model outweighs prior state-of-the-art methods. Ablation studies also show several aspects of the proposed model. Our code is available at https://github.com/chiennv2000/LR-GCN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://pypi.org/project/Wikipedia-API/

  2. https://www.sbert.net/

  3. https://www.kaggle.com/nltkdata/reuters

  4. https://arxiv.org/

References

  1. Liu J, Chang W-C, Wu Y, Yang Y (2017) Deep learning for extreme multi-label text classification. In: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval. Association for computing machinery, SIGIR ’17, pp 115–124. https://doi.org/10.1145/3077136.3080834

  2. Tang P, Jiang M, Xia BN, Pitera JW, Welser J, Chawla NV (2020) Multi-label patent categorization with non-local attention-based graph convolutional network. Proc AAAI Conf Artificial Intell 34(05):9024–9031. https://doi.org/10.1609/aaai.v34i05.6435

    Google Scholar 

  3. Huang B, Guo R, Zhu Y, Fang Z, Zeng G, Liu J, Wang Y, Fujita H, Shi Z (2022) Aspect-level sentiment analysis with aspect-specific context position information. Knowl-Based Syst 243:108473. https://doi.org/10.1016/j.knosys.2022.108473

    Article  Google Scholar 

  4. Liu W, Wang H, Shen X, Tsang I (2021) The emerging trends of multi-label learning. IEEE Trans Pattern Anal Mach Intell, pp 1–1, https://doi.org/10.1109/TPAMI.2021.3119334

  5. You R, Zhang Z, Wang Z, Dai S, Mamitsuka H, Zhu S (2019) Attentionxml: label tree-based attention-aware deep model for high-performance extreme multi-label text classification. In: Advances in neural information processing systems 32: annual conference on neural information processing systems 2019, neurIPS 2019, december 8-14, 2019, vancouver, BC, Canada, pp 5812–5822

  6. Xiao L, Zhang X, Jing L, Huang C, Song M (2021) Does head label help for long-tailed multi-label text classification. Proc AAAI Conf Artificial Intell 35(16):14103–14111

    Google Scholar 

  7. Peng H, Li J, He Y, Liu Y, Bao M, Wang L, Song Y, Yang Q (2018) Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In: Proceedings of the 2018 World Wide Web Conference. International world wide web conferences steering committee, WWW ’18, pp 1063–1072

  8. Xiao Y, Li Y, Yuan J, Guo S, Xiao Y, Li Z (2021) History-based attention in seq2seq model for multi-label text classification. Knowl-Based Syst 224:107094. https://doi.org/10.1016/j.knosys.2021.107094https://doi.org/10.1016/j.knosys.2021.107094

    Article  Google Scholar 

  9. Wang B, Hu X, Li P, Yu PS (2021) Cognitive structure learning model for hierarchical multi-label text classification. Knowl-Based Syst 218:106876. https://doi.org/10.1016/j.knosys.2021.106876

    Article  Google Scholar 

  10. Gong J, Teng Z, Teng Q, Zhang H, Du L, Chen S, Bhuiyan MZA, Li J, Liu M, Ma H (2020) Hierarchical graph transformer-based deep learning model for large-scale multi-label text classification. IEEE Access 8:30885–30896. https://doi.org/10.1109/ACCESS.2020.2972751

    Article  Google Scholar 

  11. Cai L, Song Y, Liu T, Zhang K (2020) A hybrid bert model that incorporates label semantics via adjustive attention for multi-label text classification. IEEE Access 152183-152192:8

    Google Scholar 

  12. Xiao L, Huang X, Chen B, Jing L (2019) Label-specific document representation for multi-label text classification. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). Association for computational linguistics, pp 466–475, https://doi.org/10.18653/v1/D19-1044, https://www.aclweb.org/anthology/D19-1044

  13. Huang X, Chen B, Xiao L, Yu J, Jing L (2021) Label-aware document representation via hybrid attention for extreme multi-label text classification. Neural Process Letters

  14. Chen Z-M, Wei X-S, Wang P, Guo Y (2019) Multi-label image recognition with graph convolutional networks. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR). pp 5172–5181, https://doi.org/10.1109/CVPR.2019.00532

  15. Yao L, Mao C, Luo Y (2019) Graph convolutional networks for text classification. In: In 33rd AAAI conference on artificial intelligence (AAAI-19), pp 7370–7377

  16. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. In: arXiv:1907.11692

  17. Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. In: Kok JN, Koronacki J, Mantaras RLD, Matwin S, Mladenič D, Skowron A (eds) Machine Learning: ECML 2007. Springer, pp 406–417

  18. Zhang M-L, Zhou Z-H (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048

    Article  MATH  Google Scholar 

  19. Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (Vol 1: long papers). Association for computational linguistics, Baltimore pp 655-665

  20. Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (Vol 1: long papers). Association for computational linguistics, pp 1556–1566. https://doi.org/10.3115/v1/P15-1150, https://www.aclweb.org/anthology/P15-1150

  21. Peng H, Li J, Wang S, Wang L, Gong Q, Yang R, Li B, Yu PS, He L (2021) Hierarchical taxonomy-aware and attentional graph capsule rcnns for large-scale multi-label text classification. IEEE Trans Knowl Data Eng 33(6):2505–2519

    Article  Google Scholar 

  22. Wang G, Li C, Wang W, Zhang Y, Shen D, Zhang X, Henao R, Carin L (2018) Joint embedding of words and labels for text classification. In: Proceedings of the 56th annual meeting of the association for computational linguistics (Vol 1: long papers). Association for computational linguistics, pp. 2321–2331. https://doi.org/10.18653/v1/P18-1216https://aclanthology.org/P18-1216

  23. Chai D, Wu W, Han Q, Wu F, Li J (2020) Description based text classification with reinforcement learning. In: III HD, Singh A (eds) Proceedings of the 37th international conference on machine learning. Proceedings of machine learning research. PMLR, vol 119, pp 1371–1382, https://proceedings.mlr.press/v119/chai20a.html, https://dl.acm.org/doi/10.5555/3524938.3525066, Accessed 23 March 2022

  24. Pal A, Selvakumar M, Sankarasubbu M (2020) Magnet: multi-label text classification using attention-based graph neural network. In: ICAART (2), pp 494–505

  25. Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International conference on learning representations (ICLR)

  26. Mikolov T, Chen K, Corrado Gs, Dean J (2013) Efficient estimation of word representations in vector space. Proc Workshop ICLR, vol 2013

  27. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  28. Joulin A, Grave E, Bojanowski P, Douze M, Jégou H, Mikolov T (2016) Fasttext.zip: compressing text classification models. CoRR, arXiv:1612.03651

  29. Biesialska M, Rafieian B, Costa-jussà MR (2020) Enhancing word embeddings with knowledge extracted from lexical resources. In: Proceedings of the 58th annual meeting of the association for computational linguistics: student research workshop. Association for computational linguistics, pp 271–278, https://doi.org/10.18653/v1/2020.acl-srw.36, https://aclanthology.org/2020.acl-srw.36

  30. Narayan S, Cohen SB, Lapata M (2018) Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In: Proceedings of the 2018 conference on empirical methods in natural language processing. Association for computational linguistics, pp 1797–1807, https://doi.org/10.18653/v1/D18-1206. https://aclanthology.org/D18-1206

  31. Minaee S, Kalchbrenner N, Cambria E, Nikzad N, Chenaghlu M, Gao J (2021) Deep learning–based text classification: a comprehensive review. ACM Comput Surv, vol 54(3), https://doi.org/10.1145/3439726

  32. Adhikari A, Ram A, Tang R, Lin J (2019) Docbert: Bert for document classification. In: arxiv:1904.08398

  33. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. arXiv:1907.11692

  34. Yang P, Sun X, Li W, Ma S, Wu W, Wang H (2018) SGM: Sequence generation model for multi-label classification. In: Proceedings of the 27th international conference on computational linguistics. Association for computational linguistics, pp 3915–3926, http://aclanthology.lst.uni-saarland.de/C18-1330.pdf, Accessed 23 March 2022

  35. Lewis DD, Yang Y, Rose TG, Li F (2004) Rcv1: a new benchmark collection for text categorization research. J Mach Learn Res 5:361–397

    Google Scholar 

  36. Yang Y (1999) An evaluation of statistical approaches to text categorization. Inf. Retr 1 (1–2):69–90. https://doi.org/10.1023/A:1009982220290

    Article  Google Scholar 

  37. Ionescu RT, Butnaru A (2019) Vector of locally-aggregated word embeddings (VLAWE): a novel document-level representation. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, Vol 1 (long and short papers). Association for Computational Linguistics, pp 363–369, https://doi.org/10.18653/v1/N19-1033, https://www.aclweb.org/anthology/N19-1033

Download references

Acknowledgment

This research is funded by Hung Yen University of Technology and Education under grant number UTEHY.L.2020.08.

Author information

Authors and Affiliations

  1. Hung Yen University of Technology and Education, Hung Yen, Vietnam

    Huy-The Vu, Minh-Tien Nguyen, Van-Quyet Nguyen & Van-Hau Nguyen

  2. Hanoi University of Science and Technology, Hanoi, Vietnam

    Van-Chien Nguyen

  3. Foreign Trade University, Hanoi, Vietnam

    Minh-Hieu Pham

Authors
  1. Huy-The Vu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minh-Tien Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Van-Chien Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Minh-Hieu Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Van-Quyet Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Van-Hau Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Van-Hau Nguyen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vu, HT., Nguyen, MT., Nguyen, VC. et al. Label-representative graph convolutional network for multi-label text classification. Appl Intell 53, 14759–14774 (2023). https://doi.org/10.1007/s10489-022-04106-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04106-x

Keywords

Search

Navigation