Neural Network Sentiment Classification of Russian Sentences into Four Classes

Abstract

This work is devoted to the sentiment classification of Russian sentences into four classes: positive, negative, mixed, and neutral. Unlike most modern works in this area, a mixed-sentiment class of sentences is introduced. Mixed-sentiment sentences contain both positively and negatively inflected speech at the same time. To solve this problem, the following tools were applied: an attention-based LSTM neural network, a dual attention-based GRU neural network, a and BERT neural network with several output layer modifications providing classification into four classes. Experiments comparing the effectiveness of various neural networks have been carried out on three corpora of Russian-language sentences. Two corpora are made up of users reviews: one with clothing reviews and the other with hotel reviews. The third corpus is made up of news articles from Russian publications. The best average weighted F-measure in the experiments of 0.90 is achieved by a BERT model on the clothing review corpus. The best F-measures for positive and negative sentences are noted on the same corpus, amounting to 0.92 and 0.93, respectively. The best classification results for neutral and mixed-sentiment sentences are achieved on the news corpus. For them, the F-measure is 0.72 and 0.58, respectively. As a result of the experiments, a significant superiority of BERT transfer neural networks over the previous generation LSTM and GRU neural networks is shown, which is most pronounced when classifying texts with weakly expressed sentiments. An analysis of the errors shows that “adjacent” sentiment classes (positive/negative and mixed) account for a larger proportion of errors in classification using BERT than “opposite” classes (positive and negative, neutral and mixed).