Skip to main content
SpringerLink
Log in

Multi-hypergraph Neural Networks for Emotion Recognition in Multi-party Conversations

  • Conference paper
  • First Online:
Man-Machine Speech Communication (NCMMSC 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1765))

Included in the following conference series:

  • 404 Accesses

Abstract

Emotion recognition in multi-party conversations (ERMC) is becoming increasingly popular as an emerging research topic in natural language processing. Although previous work exploited inter-dependency and self-dependency among participants, they paid more attention to the use of specific-speaker contexts. Specific-speaker context modeling can well consider the speaker’s self-dependency, but inter-dependency has not been fully utilized. In this paper, two hypergraphs are designed to model specific-speaker context and non-specific-speaker context respectively, so as to deal with self-dependency and inter-dependency among participants. To this end, we design a multi-hypergraph neural network for ERMC, namely ERMC-MHGNN. In particular, we combine average aggregation and attention aggregation to generate hyperedge features, which can make better use of utterance information. Extensive experiments are conducted on two ERC benchmarks with state-of-the-art models employed as baselines for comparison. The empirical results demonstrate the superiority of this new model and confirm that further exploiting inter-dependency is of great value for ERMC. In addition, we also achieved good results on the emotional shift issue.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chatterjee, A., Narahari, K.N., Joshi, M., Agrawal, P.: SemEval-2019 task 3: EmoContext contextual emotion detection in text. In: Proceedings of SemEval, pp. 39–48. Association for Computational Linguistics, Minneapolis, Minnesota, USA (Jun 2019). https://doi.org/10.18653/v1/S19-2005

  2. Ding, K., Wang, J., Li, J., Li, D., Liu, H.: Be more with less: Hypergraph attention networks for inductive text classification. In: Proceedings of EMNLP, pp. 4927–4936. Association for Computational Linguistics, Online (Nov 2020). https://doi.org/10.18653/v1/2020.emnlp-main.399

  3. Feng, Y., You, H., Zhang, Z., Ji, R., Gao, Y.: Hypergraph neural networks. In: Proceedings of AAAI, pp. 3558–3565. AAAI Press (2019). https://doi.org/10.1609/aaai.v33i01.33013558

  4. Ghosal, D., Majumder, N., Gelbukh, A., Mihalcea, R., Poria, S.: COSMIC: COmmonSense knowledge for eMotion identification in conversations. In: Proc. of EMNLP Findings, pp. 2470–2481. Association for Computational Linguistics, Online (Nov 2020). https://doi.org/10.18653/v1/2020.findings-emnlp.224

  5. Ghosal, D., Majumder, N., Poria, S., Chhaya, N., Gelbukh, A.: DialogueGCN: A graph convolutional neural network for emotion recognition in conversation. In: Proc. of EMNLP, pp. 154–164. Association for Computational Linguistics, Hong Kong, China (Nov 2019). https://doi.org/10.18653/v1/D19-1015

  6. Hu, D., Wei, L., Huai, X.: DialogueCRN: Contextual reasoning networks for emotion recognition in conversations. In: Proceedings of ACL, pp. 7042–7052. Association for Computational Linguistics, Online (Aug 2021). https://doi.org/10.18653/v1/2021.acl-long.547

  7. Hu, J., Liu, Y., Zhao, J., Jin, Q.: MMGCN: Multimodal fusion via deep graph convolution network for emotion recognition in conversation. In: Proceedings of ACL, pp. 5666–5675. Association for Computational Linguistics, Online (Aug 2021). https://doi.org/10.18653/v1/2021.acl-long.440

  8. Ishiwatari, T., Yasuda, Y., Miyazaki, T., Goto, J.: Relation-aware graph attention networks with relational position encodings for emotion recognition in conversations. In: Proceedings of EMNLP, pp. 7360–7370. Association for Computational Linguistics, Online (Nov 2020). https://doi.org/10.18653/v1/2020.emnlp-main.597

  9. Lee, B., Choi, Y.S.: Graph based network with contextualized representations of turns in dialogue. In: Proceedins of EMNLP, pp. 443–455. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic (Nov 2021). https://doi.org/10.18653/v1/2021.emnlp-main.36

  10. Li, J., Lin, Z., Fu, P., Wang, W.: Past, present, and future: Conversational emotion recognition through structural modeling of psychological knowledge. In: Proceedings of EMNLP Findings, pp. 1204–1214. Association for Computational Linguistics, Punta Cana, Dominican Republic (Nov 2021). https://doi.org/10.18653/v1/2021.findings-emnlp.104

  11. Li, J., Zhang, M., Ji, D., Liu, Y.: Multi-task learning with auxiliary speaker identification for conversational emotion recognition. CoRR abs/2003.01478 (2020)

    Google Scholar 

  12. Li, S., Yan, H., Qiu, X.: Contrast and generation make BART a good dialogue emotion recognizer. In: Proceedings of AAAI, pp. 11002–11010. AAAI Press (2022)

    Google Scholar 

  13. Li, Z., Tang, F., Zhao, M., Zhu, Y.: Emocaps: Emotion capsule based model for conversational emotion recognition. In: Muresan, S., Nakov, P., Villavicencio, A. (eds.) Proc. of ACL Findings. pp. 1610–1618. Association for Computational Linguistics (2022). 10.18653/v1/2022.findings-acl.126

    Google Scholar 

  14. Liu, Y.,et al.: Roberta: A robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019)

    Google Scholar 

  15. Majumder, N., Poria, S., Hazarika, D., Mihalcea, R., Gelbukh, A., Cambria, E.: Dialoguernn: An attentive rnn for emotion detection in conversations. In: Proceedings of AAAI. AAAI’19/IAAI’19/EAAI’19, AAAI Press (2019). https://doi.org/10.1609/aaai.v33i01.33016818

  16. Ong, D., et al.: Is discourse role important for emotion recognition in conversation? In: Proceedings of AAAI, pp. 11121–11129. AAAI Press (2022)

    Google Scholar 

  17. Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., Mihalcea, R.: MELD: A multimodal multi-party dataset for emotion recognition in conversations. In: Proceedings of ACL, pp. 527–536. Association for Computational Linguistics, Florence, Italy (Jul 2019). https://doi.org/10.18653/v1/P19-1050

  18. Poria, S., Majumder, N., Mihalcea, R., Hovy, E.: Emotion recognition in conversation: Research challenges, datasets, and recent advances. IEEE Access 7, 100943–100953 (2019). https://doi.org/10.1109/ACCESS.2019.2929050

    Article  Google Scholar 

  19. Shen, W., Chen, J., Quan, X., Xie, Z.: Dialogxl: All-in-one xlnet for multi-party conversation emotion recognition. In: Proceedings of AAAI (2021)

    Google Scholar 

  20. Shen, W., Wu, S., Yang, Y., Quan, X.: Directed acyclic graph network for conversational emotion recognition. In: Proceedings of ACL, pp. 1551–1560. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.acl-long.123

  21. Sun, Y., Yu, N., Fu, G.: A discourse-aware graph neural network for emotion recognition in multi-party conversation. In: Proceedings off EMNLP Findings,D pp. 2949–2958. Association for Computational Linguistics, Punta Cana, Dominican Republic (Nov 2021). https://doi.org/10.18653/v1/2021.findings-emnlp.252

  22. Vaswani, A., et al.: Attention is all you need. In: Proceedings of NeurIPS (2017)

    Google Scholar 

  23. Wang, J., Ding, K., Zhu, Z., Caverlee, J.: Session-based recommendation with hypergraph attention networks. In: Demeniconi, C., Davidson, I. (eds.) Proceedings of SDM. pp. 82–90. SIAM (2021). https://doi.org/10.1137/1.9781611976700.10

  24. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., Le, Q.V.: XLNet: Generalized Autoregressive Pretraining for Language Understanding. Curran Associates Inc., Red Hook, NY, USA (2019)

    Google Scholar 

  25. Yi, J., Park, J.: Hypergraph convolutional recurrent neural network. In: Gupta, R., Liu, Y., Tang, J., Prakash, B.A. (eds.) Proceedings of KDD, pp. 3366–3376. ACM (2020). https://doi.org/10.1145/3394486.3403389

  26. Zahiri, S.M., Choi, J.D.: Emotion detection on TV show transcripts with sequence-based convolutional neural networks. In: Proceedings of AAAI. AAAI Technical Report, vol. WS-18, pp. 44–52. AAAI Press (2018)

    Google Scholar 

  27. Zhu, J., Zhao, X., Hu, H., Gao, Y.: Emotion recognition from physiological signals using multi-hypergraph neural networks. In: Proceedings of ICME (2019). https://doi.org/10.1109/ICME.2019.00111

  28. Zhu, L., Pergola, G., Gui, L., Zhou, D., He, Y.: Topic-driven and knowledge-aware transformer for dialogue emotion detection. In: Proceedings of ACL, pp. 1571–1582. Association for Computational Linguistics, Online (Aug 2021). https://doi.org/10.18653/v1/2021.acl-long.125

Download references

Author information

Authors and Affiliations

  1. AHU -IAI AI Joint Laboratory, Anhui University, Hefei, China

    Cheng Zheng & Haojie Xu

  2. Hefei University of Technology, Hefei, China

    Xiao Sun

  3. Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China

    Haojie Xu & Xiao Sun

Authors
  1. Cheng Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haojie Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao Sun .

Editor information

Editors and Affiliations

  1. University of Science and Technology of China, Anhui, China

    Ling Zhenhua

  2. Hefei University, Anhui, China

    Gao Jianqing

  3. Shanghai Jiaotong University, Shanghai, China

    Yu Kai

  4. Tsinghua University, Beijing, China

    Jia Jia

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zheng, C., Xu, H., Sun, X. (2023). Multi-hypergraph Neural Networks for Emotion Recognition in Multi-party Conversations. In: Zhenhua, L., Jianqing, G., Kai, Y., Jia, J. (eds) Man-Machine Speech Communication. NCMMSC 2022. Communications in Computer and Information Science, vol 1765. Springer, Singapore. https://doi.org/10.1007/978-981-99-2401-1_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-2401-1_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-2400-4

  • Online ISBN: 978-981-99-2401-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation