Skip to main content
SpringerLink
Log in

Label-Dependent Hypergraph Neural Network for Enhanced Multi-label Text Classification

  • Conference paper
  • First Online:
Web Information Systems Engineering – WISE 2023 (WISE 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14306))

Included in the following conference series:

  • 926 Accesses

Abstract

Multi-label text classification (MLTC) is a challenging task in natural language processing. Improving the performance of MLTC through building label dependencies remains a focus of current research. Previous researches used label tree structure or label graph structure to build label dependencies. However, these label dependency structure building methods suffer from complexity and lack of interpretability of label relationships. To solve these problems, we propose a new model LHGN: Label-Dependent Hypergraph Neural Network for Enhanced Multi-label Text Classification, which introduces hypergraph structure to build label-dependent relationships, enhance the correlation between labels, reduce graph complexity and improve the interpretability of label relationships. In addition, we build hypergraph structures for each text instance to capture its structural information, and use the BERT model to capture the semantic information of texts. By integrating text information and combining the hypergraph label structure dependencies for multi-label text classification. Experimental results on three benchmark datasets demonstrate that the LHGN model outperforms state-of-the-art baseline models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.sbert.net/.

  2. 2.

    https://huggingface.co/bert-base-uncased.

References

  1. Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432 (2015)

    Google Scholar 

  2. Guo, L., Jin, B., Yu, R., Yao, C., Sun, C., Huang, D.: Multi-label classification methods for green computing and application for mobile medical recommendations. IEEE Access 4, 3201–3209 (2016)

    Article  Google Scholar 

  3. Feng, S., Wang, Y., Song, K., Wang, D., Yu, G.: Detecting multiple coexisting emotions in microblogs with convolutional neural networks. Cogn. Comput. 10, 136–155 (2018). https://doi.org/10.1007/s12559-017-9521-1

    Article  Google Scholar 

  4. Li, R., Si, Q., Fu, P., Lin, Z., Wang, W., Shi, G.: A multi-channel neural network for imbalanced emotion recognition. In: 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), pp. 353–360. IEEE (2019)

    Google Scholar 

  5. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

  6. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  7. Pal, A., Selvakumar, M., Sankarasubbu, M.: Multi-label text classification using attention-based graph neural network. arXiv preprint arXiv:2003.11644 (2020)

  8. Huang, X., Chen, B., Xiao, L., Yu, J., Jing, L.: Label-aware document representation via hybrid attention for extreme multi-label text classification. Neural Process. Lett. 54, 3601–3617 (2022). https://doi.org/10.1007/s11063-021-10444-7

    Article  Google Scholar 

  9. Vu, H.T., Nguyen, M.T., Nguyen, V.C., Pham, M.H., Nguyen, V.Q., Nguyen, V.H.: Label-representative graph convolutional network for multi-label text classification. Appl. Intell. 53, 14759–14774 (2023). https://doi.org/10.1007/s10489-022-04106-x

    Article  Google Scholar 

  10. Ding, K., Wang, J., Li, J., Li, D., Liu, H.: Be more with less: hypergraph attention networks for inductive text classification. arXiv preprint arXiv:2011.00387 (2020)

  11. Feng, Y., You, H., Zhang, Z., Ji, R., Gao, Y.: Hypergraph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3558–3565 (2019)

    Google Scholar 

  12. Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)

  13. Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101 (2016)

  14. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)

    Google Scholar 

  15. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  16. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  17. Yang, P., Sun, X., Li, W., Ma, S., Wu, W., Wang, H.: SGM: sequence generation model for multi-label classification. arXiv preprint arXiv:1806.04822 (2018)

  18. You, R., Zhang, Z., Wang, Z., Dai, S., Mamitsuka, H., Zhu, S.: AttentionXML: label tree-based attention-aware deep model for high-performance extreme multi-label text classification. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  19. Xiao, L., Huang, X., Chen, B., Jing, L.: Label-specific document representation for multi-label text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 466–475 (2019)

    Google Scholar 

  20. Liu, M., Liu, L., Cao, J., Du, Q.: Co-attention network with label embedding for text classification. Neurocomputing 471, 61–69 (2022)

    Article  Google Scholar 

  21. Wang, X., et al.: Heterogeneous graph attention network. In: The World Wide Web Conference, pp. 2022–2032 (2019)

    Google Scholar 

  22. Zhu, X., Zhang, Y., Zhang, Z., Guo, D., Li, Q., Li, Z.: Interpretability evaluation of botnet detection model based on graph neural network. In: IEEE INFOCOM 2022-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 1–6. IEEE (2022)

    Google Scholar 

  23. Li, I., Feng, A., Wu, H., Li, T., Suzumura, T., Dong, R.: LiGCN: label-interpretable graph convolutional networks for multi-label text classification. arXiv preprint arXiv:2103.14620 (2021)

  24. Guo, H., Li, X., Zhang, L., Liu, J., Chen, W.: Label-aware text representation for multi-label text classification. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7728–7732. IEEE (2021)

    Google Scholar 

  25. Lin, Y., et al.: BertGCN: transductive text classification by combining GCN and BERT. arXiv preprint arXiv:2105.05727 (2021)

  26. Apté, C., Damerau, F., Weiss, S.M.: Automated learning of decision rules for text categorization. ACM Trans. Inf. Syst. (TOIS) 12(3), 233–251 (1994)

    Article  Google Scholar 

  27. Loza Mencía, E., Fürnkranz, J.: Efficient pairwise multilabel classification for large-scale problems in the legal domain. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008. LNCS (LNAI), vol. 5212, pp. 50–65. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87481-2_4

    Chapter  Google Scholar 

  28. Liu, J., Chang, W.C., Wu, Y., Yang, Y.: Deep learning for extreme multi-label text classification. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 115–124 (2017)

    Google Scholar 

  29. Zhang, W., Yan, J., Wang, X., Zha, H.: Deep extreme multi-label learning. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp. 100–107 (2018)

    Google Scholar 

  30. Xiao, L., Zhang, X., Jing, L., Huang, C., Song, M.: Does head label help for long-tailed multi-label text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 14103–14111 (2021)

    Google Scholar 

  31. Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 7370–7377 (2019)

    Google Scholar 

Download references

Acknowledgments.

This work is supported by “National Key Research and Development Project (No. 2021YFF0901300)”, “Taishan Scholars Program (NO. tsqn202211203)”.

Author information

Authors and Affiliations

  1. Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China

    Xuqiang Xue, Xiaoming Wu, Shengnan Li & Xiangzhi Liu

  2. Shandong Provincial Key Laboratory of Computer Networks, Shandong Fundamental Research Center for Computer Science, Jinan, China

    Xuqiang Xue, Xiaoming Wu, Shengnan Li & Xiangzhi Liu

  3. School of IT, Deakin University, Geelong, VIC, 3220, Australia

    Man Li

Authors
  1. Xuqiang Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoming Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shengnan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiangzhi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Man Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangzhi Liu .

Editor information

Editors and Affiliations

  1. Renmin University of China, Beijing, China

    Feng Zhang

  2. Victoria University, Footscray, VIC, Australia

    Hua Wang

  3. Qatar University, Doha, Qatar

    Mahmoud Barhamgi

  4. Swinburne University of Technology, Hawthorn, Australia

    Lu Chen

  5. Swinburne University of Technology, Hawthorn, Australia

    Rui Zhou

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xue, X., Wu, X., Li, S., Liu, X., Li, M. (2023). Label-Dependent Hypergraph Neural Network for Enhanced Multi-label Text Classification. In: Zhang, F., Wang, H., Barhamgi, M., Chen, L., Zhou, R. (eds) Web Information Systems Engineering – WISE 2023. WISE 2023. Lecture Notes in Computer Science, vol 14306. Springer, Singapore. https://doi.org/10.1007/978-981-99-7254-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-7254-8_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-7253-1

  • Online ISBN: 978-981-99-7254-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation