Skip to main content
SpringerLink
Log in

Interactive capsule network for implicit sentiment analysis

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Existing sentiment analysis models mainly rely on evident emotive words within phrases. When the apparent emotional words within phrases are eliminated, the performance of these models will inevitably decrease. The implicit communication of emotion without the use of explicit emotional phrases is highly widespread in several cultures. As a result, a classification model is required to learn the link between contexts and the emotions they trigger in an automatic way. Based on whether the sentence should be segmented at the keyword position, existing methods apply either segmented or nonsegmented approaches. When emotional words are removed from a sentence, the nonsegmented approaches may lose syntactic information. To address these issues, an interactive iapsule network was proposed in this paper to extend the segmented approach. Taking the keyword as the segmented position, the network initializes two BERT models from a pretrained checkpoint with shared parameters as the encoder to process both contexts separately. By using both interactive attention and the capsule network with a dynamic routing algorithm, the model can automatically learn the insightful relationship between the former and the latter contexts. After fusing the former and latter context features, the interactive capsule network leverages both local and global attention to complete the sentiment analysis task. Experimental results on both English and Chinese corpora show that the proposed interactive attention model achieves a better performance than existing methods during implicit sentiment analysis tasks. In addition, the proposed model outperformed the top 3 models on WASSA-2018 implicit English shared tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://github.com/L-Maybe/WeiboSpider-chinese-implicit-emotion-analysis

  2. http://implicitemotions.wassa2018.com/

  3. https://nlp.stanford.edu/projects/glove/

  4. https://ai.tencent.com/ailab/nlp/embedding.html

  5. https://pypi.org/project/jieba/

  6. https://github.com/cbaziotis/ekphrasis

  7. https://github.com/huggingface/transformers

References

  1. Hudlicka E (2011) Guidelines for designing computational models of emotions. Int J Synth Emotion 2(1):26–79. https://doi.org/10.4018/jse.2011010103

    Article  Google Scholar 

  2. Kantor P (2001) Foundations of statistical natural language processing. Inf Retr 4(1):80–81. https://doi.org/10.1023/A:1011424425034

    Article  Google Scholar 

  3. Plutchik R (2001) The nature of emotions: Human emotions have deep evolutionary roots. Am Sci 89(4):344–350. https://doi.org/10.1511/2001.4.344

    Article  Google Scholar 

  4. Rao Y, Xie H, Li J, Jin F, Wang FL, Li Q (2016) Social emotion classification of short text via topic-level maximum entropy model. Inform Manag 53(8):978–986. https://doi.org/10.1016/j.im.2016.04.005

    Article  Google Scholar 

  5. Ren R, Wu DD, Wu DD (2019) Forecasting stock market movement direction using sentiment analysis and support vector machine. IEEE Syst J 13(1):760–770. https://doi.org/10.1109/JSYST.2018.2794462

    Article  Google Scholar 

  6. Song S, Wang C, Liu S, Chen H, Chen H, Bao H (2020) Sentiment Analysis Technologies in AliMe – An Intelligent Assistant for E-commerce. Int J Asian Lang Process 30(04):2050016. https://doi.org/10.1142/s2717554520500162

    Article  Google Scholar 

  7. Fadhil A, Gabrielli S (2017) Addressing challenges in promoting healthy lifestyles: The Ai-Chatbot approach. In: ACM International conference proceeding series, pp 261–265

  8. Strapparava C, Mihalcea R (2008) Learning to identify emotions in text. In: Proceedings of the ACM symposium on applied computing, pp 1556–1560

  9. Rao Y, Quan X, Wenyin L, Li Q, Chen M (2012) Building word-emotion mapping dictionary for online news. In: CEUR Workshop Proceedings, vol 917, pp 28–39

  10. Kim Y (2014) Convolutional neural networks for sentence classification. In: EMNLP 2014 - 2014 Conference on empirical methods in natural language processing, proceedings of the conference, pp 1746–1751

  11. Graves A (2012) Supervised sequence labelling with recurrent neural networks. Stud Comput Intell:1–141

  12. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  13. Mikolov T, Karafiát M, Burget L, Jan C, Khudanpur S (2010) Recurrent neural network based language model. In: Proceedings of the 11th annual conference of the international speech communication association, INTERSPEECH 2010, pp 1045– 1048

  14. Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics on human language technology, pp 2227–2237

  15. Devlin J, Chang M W, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceeding of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1, pp 4171–4186

  16. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) RoBERTa: A robustly optimized BERT pretraining approach. arXiv (1)

  17. Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: A lite bert for self-supervised learning of language representations. arXiv 1–17

  18. Du C, Sun H, Wang J, Qi Q, Liao J (2020) Adversarial and domain-aware BERT for cross-domain sentiment analysis, pp 4019–4028. https://doi.org/10.18653/v1/2020.acl-main.370

  19. Li M, Chen L, Zhao J, Li Q (2021) Sentiment analysis of Chinese stock reviews based on BERT model. Appl Intell 51(7):5016–5024. https://doi.org/10.1007/s10489-020-02101-8

    Article  Google Scholar 

  20. Liao W, Zeng B, Yin X, Wei P (2021) An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa. Appl Intell 51(6):3522–3533. https://doi.org/10.1007/s10489-020-01964-1

    Article  Google Scholar 

  21. Klinger R, De Clercq O, Mohammad S, Balahur A (2018) IEST: WASSA-2018 implicit emotions shared task. In: Proceeding of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 31–42

  22. Pantic M (2011) Affective computing. Encycloped Multimed Technol Netw:8–14. https://doi.org/10.4018/978-1-59140-561-0.ch002

  23. Přibáň P, Martínek J (2018) UWB at IEST 2018: Emotion prediction in Tweets with bidirectional long short-term memory neural network. In: Proceeding of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 224–230

  24. Witon W, Colombo P, Modi A, Kapadia M (2018) Disney at IEST 2018: Predicting emotions using an ensemble. In: the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis subjectivity, sentiment and social media analysis, pp 248–253

  25. Zhou Q, Wu H (2018) NLP at IEST 2018: BiLSTM-attention and LSTM-attention via soft voting in emotion classification. In: Proceeding of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 189–194

  26. Scherer K (2005) Handbook of Cognition and Emotion. In: Handbook of Cognition and Emotion. Wiley

  27. Alm C O, Roth D, Sproat R (2005) Emotions from text: Machine learning for text-based emotion prediction. HLT/EMNLP 2005 - Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pp 579–586

  28. Andres A J (2012) Detecting implicit expressions of emotion in text: A comparative analysis. Decis Support Syst 53(4):742–753. https://doi.org/10.1016/j.dss.2012.05.024

    Article  Google Scholar 

  29. Liu H, Singh P (2004) ConceptNet - a practical commonsense reasoning tool-kit. BT Technol J 22(4):211–226. https://doi.org/10.1023/B:BTTJ.0000047600.45421.6d

    Article  Google Scholar 

  30. Rozental A, Fleischer D, Kelrich Z (2018) Amobee at IEST 2018: Transfer learning from language models. In: Proceeding of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 43–49

  31. Jorge A, Balazs E M-T, Matsuo Y (2018) IIIDYT at IEST 2018: implicit emotion classification with deep contextualized word representations. In: Proceedings of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 50–56

  32. Alexandra Chronopoulou A M, Christos Baziotis A P (2018) NTUA-SLP at IEST 2018: ensemble of neural transfer methods for implicit emotion classification. In: Proceedings of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 57–64

  33. Paetzold G (2018) UTFPR at IEST 2018: Exploring character-to-word composition for emotion analysis. In: Proceeding of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 176–181

  34. Proisl T, Heinrich P, Kabashi B, Evert S (2018) EmotiKLUE at IEST 2018: Topic-informed classification of implicit emotions. In: Proceeding of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 235–242

  35. Zhang Y, Ji D H, Su Y, Wu H (2013) Joint naïve bayes and LDA for unsupervised sentiment analysis, vol 7818 LNAI

  36. Sezer AUB, Sezer NEA (2018) HUMIR at IEST-2018: Lexicon-sensitive and left-right context-sensitive BiLSTM for implicit emotion recognition. In: Proceeding of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 182–188

  37. Yang Q, Rao Y, Xie H, Wang J, Wang F L, Chan WH (2019) Segment-level joint topic-sentiment model for online review analysis. IEEE Intell Syst 34(1):43–50. https://doi.org/10.1109/MIS.2019.2899142

    Article  Google Scholar 

  38. Li J, Chiu B, Shang S, Shao L (2020) Neural text segmentation and its application to sentiment analysis. IEEE Trans Knowl Data Eng XX(XX):1–1. https://doi.org/10.1109/tkde.2020.2983360

    Google Scholar 

  39. Zhao G, Liu Z, Chao Y, Qian X (2020) CAPER: Context-Aware Personalized Emoji Recommendation. IEEE Trans Knowl Data Eng 4347(c):1–1. https://doi.org/10.1109/tkde.2020.2966971

    Google Scholar 

  40. Xie J Y, Ferreira Pinto Junior R, Hirst G, Xu Y (2019) Text-based inference of moral sentiment change. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp 4653–4662

  41. Chauhan D S, Akhtar M S, Ekbal A, Bhattacharyya P (2019) Context-aware interactive attention for multi-modal sentiment and emotion analysis. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp 5646–5656

  42. Ekman P (1992) An argument for basic emotions. Cogn Emotion 6:169–200

    Article  Google Scholar 

  43. Wu Y, Schuster M, Chen Z, Le Q V, Norouzi M, Macherey W, Krikun M, Cao Y, Gao Q, Macherey K, Klingner J, Shah A, Johnson M, Liu X, Kaiser L, Gouws S, Kato Y, Kudo T, Kazawa H, Stevens K, Kurian G, Patil N, Wang W, Young C, Smith J, Riesa J, Rudnick A, Vinyals O, Corrado G, Hughes M, Dean J (2016) Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv:1609.08144

  44. Vaswani A (2017) Attention is all you need. In: Proceeding of 31st Conference on Neural Information Processing Systems (NIPS 2017)

  45. Sabour S, Frosst N, Hinton G E (2017) Dynamic routing between capsules. Adv Neural Inf Process Syst 2017(Nips):3857– 3867

    Google Scholar 

  46. Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  47. Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. In: 52nd Annual meeting of the association for computational linguistics, ACL 2014 - Proceedings of the Conference, vol 1, pp 655–665

  48. Schuster M, Paliwal K K (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681. https://doi.org/10.1109/78.650093

    Article  Google Scholar 

  49. Zhang L, Wu L, Li S, Wang Z, Zhou G (2018) Cross-Lingual emotion classification with auxiliary and attention neural networks. vol 11108 LNAI

  50. Wang Q, Xu C, Zhou Y, Ruan T, Gao D, He P (2019) An attention-based BI-GRU-CapsNet model for hypernymy detection between compound entities. In: Proceedings - 2018 IEEE international conference on bioinformatics and biomedicine, BIBM 2018, pp 1031–1035

  51. Pennington J, Socher R, Manning CD (2014) GloVe: Global vectors for word representation. In: EMNLP 2014 - 2014 Conference on empirical methods in natural language processing, proceedings of the conference. https://doi.org/10.3115/v1/d14-1162, https://www.aclweb.org/anthology/D14-1162/, pp 1532–1543

  52. Prechelt L (1998) Automatic early stopping using cross validation: quantifying the criteria. Neural Netw 11(4):761–767. https://doi.org/10.1016/S0893-6080(98)00010-0

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (NSFC) under Grant Nos. 61702443, 61966038, and 61762091. The authors would like to thank the anonymous reviewers for their constructive comments.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yunnan University, Kunming, China

    Yanjun Qian, Jin Wang, Dawei Li & Xuejie Zhang

Authors
  1. Yanjun Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dawei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuejie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin Wang.

Ethics declarations

Conflict of Interests

We would like to submit the enclosed manuscript entitled “Interactive Capsule Network for Implicit Sentiment Analysis”, which we wish to be considered for publication in “Applied Intelligence”. No conflict of interest exists in the submission of this manuscript, and the manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously, and is not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The code of this paper is available at: https://github.com/kongfangyi/InteractiveCapsuleNetwork

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qian, Y., Wang, J., Li, D. et al. Interactive capsule network for implicit sentiment analysis. Appl Intell 53, 3109–3123 (2023). https://doi.org/10.1007/s10489-022-03584-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03584-3

Keywords

Search

Navigation