A Comparative Study of Publicly Available Russian Sentiment Lexicons

  • Evgeny KotelnikovEmail author
  • Tatiana Peskisheva
  • Anastasia Kotelnikova
  • Elena Razova
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 930)


Sentiment lexicons play an important role in the systems of sentiment analysis and opinion mining. The article takes a look into eight publicly available Russian sentiment lexicons of today. A joint analysis of these lexicons was done by finding unions and intersections of the lexicons and also analysing the distribution of parts of speech. In order to study the quality of the lexicons, a sentiment classification is made based on the SVM and the TF-IDF model. Text corpora from reviews of works of art (books and movies), organizations (banks and hotels) and goods (kitchen appliances) are made for this purpose. Lexicons are compared in terms of their classification quality, and also on the basis of a linear regression model that reflects the dependence of their F1-measure on their TF-IDF model size. The resulting union lexicon most fully reflects the sentiment lexica of the present day Russian language and can be used both in scientific research and in applied sentiment analysis systems.


Sentiment lexicons Sentiment analysis Opinion mining 



This work was carried out as a part of the project of Government Order No. 34.2092.2017/4.6 of the Ministry of Education and Science of the Russian Federation.


  1. 1.
    Baccianella, S., Esuli, A., Sebastiani, F.: SentiWordNet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: Proceedings of the Seventh Conference on International Language Resources and Evaluation (LREC 2010), Valletta, pp. 2200–2204 (2010)Google Scholar
  2. 2.
    Balahur, A., Hermida, J.M., Montoyo, A.: Detecting implicit expressions of emotion in text: a comparative analysis. Decis. Support Syst. 53, 742–753 (2012)CrossRefGoogle Scholar
  3. 3.
    Blinov, P.D., Klekovkina, M.V., Kotelnikov, E.V., Pestov, O.A.: Research of lexical approach and machine learning methods for sentiment analysis. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue-2013”, vol. 12, no. 19, pp. 51–61 (2013)Google Scholar
  4. 4.
    Chen, Y., Skiena, S.: Building sentiment lexicons for all major languages. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, pp. 383–389 (2014)Google Scholar
  5. 5.
    Chetviorkin I., Loukachevitch N.: Extraction of Russian sentiment lexicon for product meta-domain. In: Proceedings of COLING 2012, Mumbai, pp. 593–610 (2012)Google Scholar
  6. 6.
    Habernal, I., Ptáček, T., Steinberger, J.: Supervised sentiment analysis in Czech social media. Inf. Process. Manag. 51(4), 532–546 (2015)CrossRefGoogle Scholar
  7. 7.
    Hailong, Z., Wenyan, G., Bo, J.: Machine learning and lexicon based methods for sentiment classification: a survey. In: Proceedings of the 11th Web Information System and Application Conference, Tianjin, pp. 262–265 (2014)Google Scholar
  8. 8.
    Hemmatian, F., Sohrabi, M.K.: A survey on classification techniques for opinion mining and sentiment analysis. Artif. Intell. Rev. 1–51 (2017)Google Scholar
  9. 9.
    Kiselev, Y., Braslavski, P., Menshikov, I., Mukhin, M., Krizhanovskaya, N.: Russian Lexicographic landscape: a tale of 12 dictionaries. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue-2015”, vol. 14, no. 21, pp. 254–271 (2015)Google Scholar
  10. 10.
    Koltsova, O.Yu., Alexeeva, S.V., Kolcov, S.N.: An opinion word lexicon and a training dataset for russian sentiment analysis of social media. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue-2016”. vol. 15, no. 22, pp. 277–287 (2016)Google Scholar
  11. 11.
    Kotelnikov, E., Bushmeleva, N., Razova, E., Peskisheva, T., Pletneva, M.: Manually created sentiment lexicons: research and development. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue-2016”, vol. 15, no. 22, pp. 300–314 (2016)Google Scholar
  12. 12.
    Linguistic Inquiry and Word Counts. Accessed 20 May 2018
  13. 13.
    Liu, B.: Opinion Mining, Sentiment Analysis, and Opinion Spam Detection. Accessed 20 May 2018
  14. 14.
    Loukachevitch, N., Dobrov, B.: RuThes linguistic ontology vs. russian wordnets. In: Proceedings of the 7th Global Wordnet Conference (GWC 2014), Tartu, pp. 154–162 (2014)Google Scholar
  15. 15.
    Loukachevitch, N., Levchik, A.: Creating a general russian sentiment lexicon. In: Proceedings of Language Resources and Evaluation Conference LREC-2016, pp. 1171–1176 (2016)Google Scholar
  16. 16.
    Mohammad, S.M., Turney, D.P.: Crowdsourcing a word-emotion association lexicon. Comput. Intell. 29(3), 436–465 (2013)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Ohana, B., Tierney, B., Delany, S.-J.: Domain independent sentiment classification with many lexicons. In: 2011 IEEE Workshops of International Conference on Advanced Information Networking and Applications (WAINA), Singapore, pp. 632–637 (2011)Google Scholar
  18. 18.
    Pedregosa, et al.: Scikit-learn: machine learning in python. JMLR 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Potts, Ch.: Sentiment symposium tutorial: lexicons. In: Sentiment Analysis Symposium, San Francisco, 8–9 November 2011 (2011)Google Scholar
  20. 20.
    Qiu, G., Liu, B., Bu, J., Chen, C.: Opinion word expansion and target extraction through double propagation. Comput. Linguist. 37(1), 9–27 (2011)CrossRefGoogle Scholar
  21. 21.
    Saif, H., He, Y., Fernandez, M., Alani, H.: Contextual semantics for sentiment analysis of Twitter. Inf. Process. Manag. 52(1), 5–19 (2016)CrossRefGoogle Scholar
  22. 22.
    Stone, P.J., Dunphy, D.C., Smith, M.S., Ogilvie, D.M.: The General Inquirer: A Computer Approach to Content Analysis. MIT Press Cambridge, Cambridge (1966)Google Scholar
  23. 23.
    Taboada, M.: Sentiment analysis: an overview from linguistics. Annu. Rev. Linguist. 2, 325–347 (2016)CrossRefGoogle Scholar
  24. 24.
    Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M.: Lexicon-based methods for sentiment analysis. Comput. Linguist. 37(2), 267–307 (2011)CrossRefGoogle Scholar
  25. 25.
    Tutubalina, E.V.: Metody izvlecheniya i rezyumirovaniya kriticheskih otzyvov pol’zovatelej o produkcii (Extraction and summarization methods for critical user reviews of a product). Kazan Federal University, Kazan (2016)Google Scholar
  26. 26.
    Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the 2005 Human Language Technology Conference and the Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP-05), Vancouver, pp. 347–354 (2005)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Evgeny Kotelnikov
    • 1
    Email author
  • Tatiana Peskisheva
    • 1
  • Anastasia Kotelnikova
    • 1
  • Elena Razova
    • 1
  1. 1.Vyatka State UniversityKirovRussia

Personalised recommendations