Emotion Classification in Russian: Feature Engineering and Analysis

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12602)


In this paper, we address the issue of identifying emotions in Russian informal text messages. For this purpose, a new large dataset of text messages from the most popular Russian messaging/social networking services (Telegram, VK) was compiled semi-automatically. Emojis contained in the text messages were used to annotate the data for emotions expressed. This paper proposes an integrated approach to text-based emotion classification combining linguistic methods and machine learning. This approach relies on morphological, lexical, and stylistic features of the text. Furthermore, the level of expressiveness was considered as well. As a result, an emotion classification model demonstrating near-human performance was designed. In this paper, we also report on the importance of different linguistic features of the text messages for the task of automatic emotive analysis. Additionally, we perform error analysis and discover ways to improve the model in the future.


Machine learning Emotion identification Emotiveness Sentiment analysis Natural language processing 



The work is supported by RSF (Russian Science Foundation) grant 20-71-10010.


  1. 1.
    Abdul-Mageed, M., Ungar, L.: Emonet: Fine-grained emotion detection with gated recurrent neural networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long papers, pp. 718–728 (2017)Google Scholar
  2. 2.
    Canales, L., Strapparava, C., Boldrini, E., Martnez-Barco, P.: Exploiting a bootstrapping approach for automatic annotation of emotions in texts. In: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 726–734. IEEE (2016)Google Scholar
  3. 3.
    Chatterjee, A., Narahari, K.N., Joshi, M., Agrawal, P.: Semeval-2019 task 3: emocontext contextual emotion detection in text. In: Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 39–48 (2019)Google Scholar
  4. 4.
    Ekman, P.: Expression and the nature of emotion. Approaches Emot. 3(19), 344 (1984)Google Scholar
  5. 5.
    Gaind, B., Syal, V., Padgalwar, S.: Emotion detection and analysis on social media (2019). arXiv preprint arXiv:1901.08458.
  6. 6.
    Gudovskikh, D.V., Moloshnikov, I.A., Rybka, R.B.: Analiz emotivnosti tekstov na osnove psikholingvisticheskikh markerov s opredeleniem morfologicheskikh svoystv. Vestnik Voronezhskogo gosudarstvennogo universiteta. Seriya: Lingvistika i mezhkul'turnaya kommunikatsiya (3) (2015)Google Scholar
  7. 7.
    Gupta, U., Chatterjee, A., Srikanth, R., Agrawal, P.: A sentiment-and-semantics-based approach for emotion detection in textual conversations (2017). arXiv preprint arXiv:1707.06996.
  8. 8.
    Hasan, M., Rundensteiner, E., Agu, E.: Automatic emotion detection in text streams by analyzing twitter data. Int. J. Data Sci. Anal. 7(1), 35–51 (2019)CrossRefGoogle Scholar
  9. 9.
    Klinger, R.: An analysis of annotated corpora for emotion classification in text. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 2104–2119 (2018)Google Scholar
  10. 10.
    Babii, A., Kazyulina, M., Malafeev. A.: Automatic emotion identification in russian text messages. In: Computational Linguistics and Intellectual Technologies. Papers from the Annual International Conference “Dialogue”, no. 19, Supplementary volume, pp. 1002–1010 (2020)Google Scholar
  11. 11.
    Mehendale, N.: Facial emotion recognition using convolutional neural networks (FERC). SN Appl. Sci. 2(3), 1–8 (2020)CrossRefGoogle Scholar
  12. 12.
    Mohammad, S.M., Kiritchenko, S.: Using hashtags to capture fine emotion categories from tweets. Comput. Intell. 31(2), 301–326 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Mohammad, S.M., Zhu, X., Kiritchenko, S., Martin, J.: Sentiment, emotion, purpose, and style in electoral tweets. Inf. Process. Manag. 51(4), 480–499 (2015)CrossRefGoogle Scholar
  14. 14.
    Pazel'skaya, A.G., Solov'ev, A.N.: Metod opredeleniya emotsiy v tekstakh na russkom yazyke. In Komp'yuternaya lingvistika i intellektual'nye tekhnologii: Po materialam yezhegodnoy Mezhdunarodnoy konferentsii «Dialog» (Bekasovo, 25–29 maya 2011 g.). M.: Izd-vo RGGU, no. 10, p. 17 (2011)Google Scholar
  15. 15.
    Seyeditabari, A., Tabari, N., Gholizadeh, S., Zadrozny, W.: Emotion detection in text: focusing on latent representation (2019). arXiv preprint arXiv:1907.09369.
  16. 16.
    Zahiri, S.M., Choi, J.D.: Emotion detection on tv show transcripts with sequence-based convolutional neural networks. In: Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.National Research University Higher School of EconomicsNizhny NovgorodRussia

Personalised recommendations