Abstract
This paper describes the experience from carrying out open evaluation of the methods for the sentiment analysis in Russian based on the ROMIP seminar in 2011–2012. Several training collections, which are now in free access, have been created during the conduction of the track. This paper reviews the current state of the art in processing subjective texts in Russian and describes the major tasks and characteristics of collections, as well as quality-assessment measures.
Similar content being viewed by others
References
Abdul-Mageed, M., Diab, M., Korayem, M., Subjectivity and sentiment analysis of modern standard arabic, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 2011, pp. 587–591.
Amigo, E., Corujo, A., Gonzalo, J., Meij, E., Rijke, Md., Overview of replab 2012: Evaluating online reputation management systems, Proceedings of the CLEF 2012 Labs and Workshop Notebook Papers, 2012, pp. 1–24.
Awadallah, R., Ramanath, M., Weikum, G., PolariCQ: Polarity classification of political quotations, Proceedings of CIKM-2012, 2012., pp. 1945–1949.
Balasubramanyan, R., Cohen, W., Pierce, D., Redlawsk D., Modeling polarizing topics: When do different political communities respond differently to the same news? Proceedings of ICWSM, 2012.
Blinov, P., Klekovkina, M., Kotelnikov, E., Pestov, O., Research of lexical approach and machine learning methods for sentiment analysis, Proceedings of Dialog, vol. 2, 2013, pp. 51–61.
Chetviorkin, I., Braslavskiy, P., Loukachevich, N., Sentiment analysis track at ROMIP 2011, Proceedings of International Conference Dialog-2012, vol. 2, 2012, pp. 1–14.
Chetviorkin, I., Loukachevitch, N., Extraction of Russian sentiment lexicon for product meta-domain, Proceedings of COLING 2012, 2012, pp. 593–610.
Chetvirokin, I., Loukachevitch, N., Sentiment analysis track at ROMIP 2012, Proceedings of International Conference Dialog-2013, vol. 2, 2013, pp. 40–50.
Choi, Y., Cardie, C., Adapting a polarity lexicon using integer linear programming for domain-specific sentiment classification, Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2009, pp. 590–598.
Ermakov, A.E., Extraction of knowledge from a text and its processing: the state and prospects, Inform. Tekhnol., no. 7, 2009, pp. 50–55.
Kotel’nikov, E.V., Klekovkina, M.V., Automated Text Sentiment Analysis Based on Machine Learning Methods. Computer Linguistics and Intelligent Technologies: According to the Materials of the Yearly International Conference Dialog, 2012, vol. 2, pp. 27–36.
Kuznetsova, E.S., Loukachevitch, N.V., Chetviorkin, I.I., Testing rules for sentiment analysis system. Computational Linguistics and Intellectual Technologies, Proc. of International Conference Dialog-2013, vol. 2, 2013, pp.71–80.
Macdonald, C., Ounis, I., Soboroff, I., Overview of the TREC 2007 blog track, Proceedings of TREC-2007, Gaithersburg, USA, 2008.
Macdonald, C., Ounis, I., Soboroff, I., Overview of the TREC 2009 blog track, Proceedings of TREC-2009, Gaithersburg, USA, 2010.
Manning, C.D., Raghavan, P., Schiitze, H., Introduction to information retrieval, Cambridge: Cambridge University Press, 2008.
Mihalcea, R., Banea, C., Wiebe, J., Learning multilingual subjective language via cross-lingual projections, Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic, 2007, pp. 976–983.
Morante, R., Blanco, E., SEM 2012 shared task: Resolving the scope and focus of negation, Proceedings of the First Joint Conference on Lexical and Computational Semantics, Montreal, 2012, pp. 265–274
Pak, A., Paroubek, P., Language independent approach to sentiment analysis (LIMSI Participation in ROMIP’ll) Computational Linguistics and Intellectual Technologies, Proc. of International Conference Dialog-2012, vol. 2, 2012, pp. 37–50.
Pan, S.J., Ni, X., Sun, J.-T., Yang, Q., Chen, Z., Cross-domain sentiment classification via spectral feature alignment, Proceedings of the World Wide Web Conference, 2010, pp. 751–760.
Pang, B., Lee, L., Opinion mining and sentiment analysis. Foundations and Trends-in Information Retrieval. Now Publishers, 2008.
Panicheva, P. The Rule-Based ATEX Sentiment Analysis System Based during Processing Texts of Different Themes. Computer Linguistics and Intelligent Technologies: According to the Materials of the Yearly International Conference Dialog, 2013, vol. 2, pp. 101–113.
Pazal’skaya, A.G., Solovyov, A.N., The Sentiment Recognition Method for Texts in Russian. Computer Linguistics and Intelligent Technologies: According to the Materials of the Yearly International Conference Dialog, 2011, pp. 510–522.
Perez-Rosas, V., Banea, C, Mihalcea, R., Learning sentiment lexicons in Spanish, Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), 2012.
Polyakov, P.Yu., Kalinina, M.V., Pleshko, V.V. Study of the Applicability of the Thematic Classification Methods in the Task of Classifying Reports about Books. Computer Linguistics and Intelligent Technologies: According to the Materials of the Yearly International Conference Dialog, 2012, vol. 2, pp. 51–59.
Pestian, J., Matykiewicz, P., Linn-Gust, M., Sentiment analysis of suicide notes: A shared task. Biomedical Informatics Insights. 2012; 5(Suppl. 1), 2012, pp. 3–16.
Ounis, L., de Rijke, M., Macdonald, C, Mishne, G., Soboroff, I., Overview of TREC-2006 Blog track, Proceedings of TREC-2006, Gaithersburg, USA, 2007.
Ounis, I., Macdonald, C., Soboroff, I., Overview of the TREC 2008 blog track, Proceedings of TREC-2008, Gaithersburg, USA, 2009.
Ounis, I., Macdonald, C., Soboroff, I., Overview of the TREC 2010 blog track, Proceedings of TREC-2010, Gaithersburg, USA, 2011.
Seki, Y., Evans, D., Ku. L., Chen, H., Kando, N., Lin, C., Overview of opinion analysis pilot task at NTCIR-6, Proceedings of NTCIR-6 Workshop Meeting, 2007, pp. 265–278.
Steinberger, J., Lenkova, P., Ebrahim, M., Ehrmann, M., Hurriyetogly, A., Kabadjov, M., Steinberger, R., Tanev, H., Zavarella, V., Vazquez, S., Creating sentiment dictionaries via triangulation, Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis, ACL-HLT, 2011, pp. 28–36.
Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M., Lexicon-based methods for Sentiment Analysis, Comp. Ling., 2011, vol., 37(2), pp. 267–307.
Wu, Y., Jin, P., Semeval-2010 task 18: Disambiguating sentiment ambiguous adjectives, Proceedings of the 5th International Workshop on Semantic Evaluation, 2010, pp. 81–85.
Zagibalov, T., Belyatskaya, K., Carroll, J. Comparable English-Russian book review corpora for sentiment analysis, Proceedings of the 1st Workshop on Computational Approaches to Subjectivity and Sentiment Analysis WASSA, 2010, pp. 67–72.
Author information
Authors and Affiliations
Corresponding author
Additional information
Original Russian Text © N.V. Loukachevitch, I.I. Chetviorkin, 2014, published in Iskusstvennyi Intellekt i Prinyatie Reshenii, 2014, No. 1, pp. 25–33.
About this article
Cite this article
Loukachevitch, N.V., Chetviorkin, I.I. Open evaluation of sentiment-analysis systems based on the material of the Russian language. Sci. Tech.Inf. Proc. 41, 370–376 (2014). https://doi.org/10.3103/S0147688214060057
Published:
Issue Date:
DOI: https://doi.org/10.3103/S0147688214060057