Skip to main content
SpringerLink
Log in

A multichannel embedding and arithmetic optimized stacked Bi-GRU model with semantic attention to detect emotion over text data

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Emotion analysis of social media data is essential for understanding a person’s mindset and judgment on a particular event. The emotion detection framework needs sophisticated deep learning models such as Long short-term memory networks and convolutional neural networks (CNN) for their development. These models did not give adequate emotional information in targeted emotion analysis. This paper proposes a new deep learning-based emotion detection framework using an optimized, stacked, bidirectional Gated Recurrent Unit with sematic attention (SRBi-GRU-SA). A multichannel word embedding model is proposed for representing the text document by combining the emotional information with the semantic information obtained from a natural language model (BERT), Term-frequency inverse gravity moment (TF-IGM) based term weighted word embedding model with trigrams. The proposed SRBi-GRU-SA includes semantic attention (SA) model for highlighting the most significant features based on attention score. Also, an Arithmetic Optimization Algorithm (AOA) is introduced to optimize the weight parameters of the SRBi-GRU-SA model. The evaluation results of the proposed optimized SRBi-GRU-SA model prove its effectiveness against the existing models by increasing the F1 score to 84.51%, 85.93%, 85.25%, and 87.12% on International Survey on Emotion Antecedents and Reactions (ISEAR), Stance Sentiment Emotion Corpus (SSEC), Daily dialogs, and Grounded-Emotions datasets respectively

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Suhasini M, Srinivasu B (2020) Emotion detection framework for twitter data using supervised classifiers. In: Data engineering and communication technology, pp 565–576. https://doi.org/10.1007/978-981-15-1097-7_47

  2. Shoumy NJ, Ang LM, Seng KP, Rahaman DM, Zia T (2020) Multimodal big data affective analytics: a comprehensive survey using text, audio, visual and physiological signals. J Netw Comput Appl 149:102447. https://doi.org/10.1016/j.jnca.2019.102447

    Article  Google Scholar 

  3. Alswaidan N, Menai MEB (2020) A survey of state-of-the-art approaches for emotion recognition in text. Knowl Inf Syst 62(8):2937–2987. https://doi.org/10.1007/s10115-020-01449-0

    Article  Google Scholar 

  4. Kušen E, Strembeck M, Conti M (2018) Emotional valence shifts and user behavior on Twitter, Facebook, and YouTube. In: IEEE/ACM international conference on advances in social networks analysis and mining, pp 63–83. https://doi.org/10.1007/978-3-030-02592-2_4

  5. Shaukat Z, Zulfiqar AA, Xiao C, Azeem M, Mahmood M (2020) Sentiment analysis on IMDB using lexicon and neural networks. SN Appl Sci 2(2):1–10. https://doi.org/10.1007/s42452-019-1926-x

    Article  Google Scholar 

  6. Kumar A, Beri T, Sobti T (2021) A survey of sentiment analysis and opinion mining. In: Emerging technologies in data mining and information security, pp 407–416. https://doi.org/10.1007/978-981-33-4367-2_39

  7. Pradhan A, Senapati MR, Sahu PK (2021) Improving sentiment analysis with learning concepts from concept, patterns lexicons and negations. Ain Shams Eng J 13(2):101559. https://doi.org/10.1016/j.asej.2021.08.004

    Article  Google Scholar 

  8. Sailunaz K, Dhaliwal M, Rokne J, Alhajj R (2018) Emotion detection from text and speech: a survey. Soc Netw Anal Min 8(1):1–26. https://doi.org/10.1007/s13278-018-0505-2

    Article  Google Scholar 

  9. Mao X, Chang S, Shi J, Li F, Shi R (2019) Sentiment-aware word embedding for emotion classification. Appl Sci 9(7):1334. https://doi.org/10.3390/app9071334

    Article  Google Scholar 

  10. Angel Deborah S, Mirnalinee TT, Rajendram SM (2021) Emotion analysis on text using multiple Kernel Gaussian. Neural Process Letts 53(2):1187–1203. https://doi.org/10.1007/s11063-021-10436-7

    Article  Google Scholar 

  11. Osman A, Ahmad S (2019) Current trends and research directions in the dictionary-based approach for sentiment lexicon generation: a survey. J Theor Appl Inf Technol 97(22)

  12. Grljević O, Bošnjak Z, Kovačević A (2022) Opinion mining in higher education: a corpus-based approach. Enterp Inf Syst 16(5):1–26. https://doi.org/10.1080/17517575.2020.1773542

    Article  Google Scholar 

  13. Wang X, Kou L, Sugumaran V, Luo X, Zhang H (2020) Emotion correlation mining through deep learning models on natural language text. IEEE Trans Cybern 51(9):4400–4413. https://doi.org/10.1109/TCYB.2020.2987064

    Article  Google Scholar 

  14. Erenel Z, Adegboye OR, Kusetogullari H (2020) A new feature selection scheme for emotion recognition from text. Appl Sci 10(15):5351. https://doi.org/10.3390/app10155351

    Article  Google Scholar 

  15. Yadav A, Jha CK, Sharan A, Vaish V (2020) Sentiment analysis of financial news using unsupervised approach. Procedia Comput Sci 167:589–598. https://doi.org/10.1016/j.procs.2020.03.325

    Article  Google Scholar 

  16. Fernández-Gavilanes M, Juncal-Martínez J, García-Méndez S, Costa-Montenegro E, González-Castaño FJ (2018) Creating emoji lexica from unsupervised sentiment analysis of their descriptions. Expert Syst Appl 103:74–91. https://doi.org/10.1016/j.eswa.2018.02.043

    Article  Google Scholar 

  17. Asghar MZ, Khan A, Bibi A, Kundi FM, Ahmad H (2017) Sentence-level emotion detection framework using rule-based classification. Cogn Comput 9(6):868–894. https://doi.org/10.1007/s12559-017-9503-3

    Article  Google Scholar 

  18. Ahmad Z, Jindal R, Ekbal A, Bhattachharyya P (2020) Borrow from rich cousin: transfer learning for emotion detection using cross lingual embedding. Expert Syst Appl 139:112851. https://doi.org/10.1016/j.eswa.2019.112851

    Article  Google Scholar 

  19. Rathnayaka P, Abeysingh, S, Samarajeewa C, Manchanayake I, Walpola MJ, Nawaratne R, Bandaragoda T, Alahakoon D (2019) Gated recurrent neural network approach for multilabel emotion detection in microblogs. https://doi.org/10.48550/arXiv.1907.07653

  20. Gan C, Wang L, Zhang Z, Wang Z (2020) Sparse attention based separable dilated convolutional neural network for targeted sentiment analysis. Knowl Based Syst 188:104827. https://doi.org/10.1016/j.knosys.2019.06.035

    Article  Google Scholar 

  21. Wei J, Liao J, Yang Z, Wang S, Zhaow Q (2020) BiLSTM with multi-polarity orthogonal attention for implicit sentiment analysis. Neurocomputing 383:165–173. https://doi.org/10.1016/j.neucom.2019.11.054

    Article  Google Scholar 

  22. Abdul-Mageed M, Ungar L (2017) Emonet: fine-grained emotion detection with gated recurrent neural networks. In: Proceedings of the 55th annual meeting of the association for computational linguistics, pp 718–728. https://doi.org/10.18653/v1/P17-1067

  23. Colnerič N, Demšar J (2018) Emotion recognition on twitter: comparative study and training a unison model. IEEE Trans Affect Comput 11(3):433–446. https://doi.org/10.1109/TAFFC.2018.2807817

    Article  Google Scholar 

  24. Kratzwald B, Ilić S, Kraus M, Feuerriegel S, Prendinger H (2018) Deep learning for affective computing: text-based emotion recognition in decision support. Decis Support Syst 115:24–35. https://doi.org/10.1016/j.dss.2018.09.002

    Article  Google Scholar 

  25. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. https://doi.org/10.48550/arXiv.1301.3781

  26. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 26. https://doi.org/10.48550/arXiv.1310.4546

  27. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543. https://doi.org/10.3115/v1/D14-1162

  28. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606–615. https://doi.org/10.18653/v1/D16-1058

  29. Seal D, Roy UK, Basak R (2020) Sentence-level emotion detection from text based on semantic rules. In: Information and communication technology for sustainable development, pp 423–430. https://doi.org/10.1007/978-981-13-7166-0_42

  30. Bojanowski P, Grave E, Joulin A, Mikolov T (2017) Enriching word vectors with subword information. Trans Assoc Comput Linguist 5:135–146. https://doi.org/10.1162/tacl_a_00051

    Article  Google Scholar 

  31. Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. https://doi.org/10.48550/arXiv.1810.04805

  32. Sundaram V, Ahmed S, Muqtadeer SA, Reddy RR (2021) Emotion analysis in text using TF-IDF. In: 2021 11th international conference on cloud computing, data science & engineering, pp 292–297. https://doi.org/10.1109/Confluence51648.2021.9377159

  33. Debole F, Sebastiani F (2004) Supervised term weighting for automated text categorization. In: Text mining and its applications, pp 81–97. https://doi.org/10.1007/978-3-540-45219-5_7

  34. Chen K, Zhang Z, Long J, Zhang H (2016) Turning from TF-IDF to TF-IGM for term weighting in text classification. Expert Syst Appl 66:245–260. https://doi.org/10.1016/j.eswa.2016.09.009

    Article  Google Scholar 

  35. Gordeev D (2016) Detecting state of aggression in sentences using CNN. In: International conference on speech and computer, pp 240–245. https://doi.org/10.1007/978-3-319-43958-7_28

  36. Felbo B, Mislove A, Søgaard A, Rahwan I, Lehmann S (2017) Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. https://doi.org/10.48550/arXiv.1708.00524

  37. Xu G, Meng Y, Qiu X, Yu Z, Wu X (2019) Sentiment analysis of comment texts based on BiLSTM. In: Ieee ACCESS, 7, pp 51522–51532. https://doi.org/10.1109/ACCESS.2019.2909919

  38. Agrawal A, An A, Papagelis M (2018) Learning emotion-enriched word representations. In: Proceedings of the 27th international conference on computational linguistics, pp 950–961. https://aclanthology.org/C18-1081

  39. Zhou C, Sun C, Liu Z, Lau F (2015) A C-LSTM neural network for text classification. https://arxiv.org/abs/1511.08630

  40. Oberländer LAM, Klinger R (2018) An analysis of annotated corpora for emotion classification in text. In: Proceedings of the 27th international conference on computational linguistics, pp 2104–2119. https://aclanthology.org/C18-1179

  41. Shrivastava K, Kumar S, Jain DK (2019) An effective approach for emotion detection in multimedia text data using sequence based convolutional neural network. Multimed Tools Appl 78(20):29607–29639. https://doi.org/10.1007/s11042-019-07813-9

    Article  Google Scholar 

  42. Batbaatar E, Li M, Ryu KH (2019) Semantic-emotion neural network for emotion recognition from text. IEEE Access 7:111866–111878. https://doi.org/10.1109/ACCESS.2019.2934529

    Article  Google Scholar 

  43. Luong MT, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. https://doi.org/10.48550/arXiv.1508.04025

  44. Chen H, Sun M, Tu C, Lin Y, Liu Z(2016) Neural sentiment classification with user and product attention. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 1650–1659. https://doi.org/10.18653/v1/D16-1171

  45. Keerthi Kumar HM, Harish BS (2018) Classification of short text using various pre-processing techniques: an empirical evaluation. In: Recent findings in intelligent computing techniques, pp 19–30. https://doi.org/10.1007/978-981-10-8633-5_3

  46. Wikipedia. List of Emoticons (2017) http://en.wikipedia.org/wiki/List_of_emoticons

  47. Agrawal T, Singhal A (2019) An efficient knowledge-based text pre-processing approach for Twitter and Google+. In: International conference on advances in computing and data science, pp 379–389. https://doi.org/10.1007/978-981-13-9942-8_36

  48. Arora M, Kansal V (2019) Character level embedding with deep convolutional neural network for text normalization of unstructured data for Twitter sentiment analysis. Soc Netw Anal Min 9(1):1–14. https://doi.org/10.1007/s13278-019-0557-y/

    Article  Google Scholar 

  49. Scherer KR, Wallbott HG (1994) Evidence for universality and cultural variation of differential emotion response patterning. J Pers Soc Psychol 66(2):310. https://doi.org/10.1037/0022-3514.66.2.310

    Article  Google Scholar 

  50. Pradhan A, Senapati MR, Sahu PK (2022) Comparative analysis of lexicon-based emotion recognition of text. In: The third international conference on machine learning, image processing, network security and data sciences

  51. Abualigah L, Diabat A, Mirjalili S, Abd Elaziz M, Gandomi AH (2021) The arithmetic optimization algorithm. Comput Methods Appl Mech Eng 376:113609. https://doi.org/10.1016/j.cma.2020.113609

    Article  MathSciNet  MATH  Google Scholar 

  52. Schuff H, Barnes J, Mohme J, Padó S, Klinger R (2017) Annotation, modelling and analysis of fine-grained emotions on a stance and sentiment detection corpus. In: Proceedings of the 8th workshop on computational approaches to subjectivity, sentiment and social media analysis, pp 13–23. https://doi.org/10.18653/v1/W17-5203

  53. Li Y, Su H, Shen X, Li W, Cao Z, Niu S (2017) Dailydialog: a manually labelled multi-turn dialogue dataset. arXiv preprint arXiv:1710.03957. https://arxiv.org/abs/1710.03957

  54. Dietterich TG (1998) Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 10(7):1895–1923. https://doi.org/10.1162/089976698300017197

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Veer Surendra Sai University of Technology, Burla, India

    Anima Pradhan, Manas Ranjan Senapati & Pradip Kumar Sahu

Authors
  1. Anima Pradhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manas Ranjan Senapati
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pradip Kumar Sahu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manas Ranjan Senapati.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pradhan, A., Ranjan Senapati, M. & Sahu, P.K. A multichannel embedding and arithmetic optimized stacked Bi-GRU model with semantic attention to detect emotion over text data. Appl Intell 53, 7647–7664 (2023). https://doi.org/10.1007/s10489-022-03907-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03907-4

Keywords

Search

Navigation