Abstract
Sentiment analysis has become an essential and fundamental task in natural language processing that has many useful applications. There exist various methods for solving the sentiment classification problem using traditional machine learning models, deep neural networks, and transfer learning methods. It is still interesting for Vietnamese sentiment classification problems to understand the impact of different ensemble techniques and deep learning approaches for building the most suitable model. This work aims to study the Vietnamese sentiment classification on one public dataset used in the Vietnamese Sentiment Analysis Challenge 2019 and another large-scale dataset, namely “AISIA-Sent-002”, which was collected from the Vietnamese e-commerce websites by ourselves. We explore five distinct ensemble schemes, including the classic methods (Uniform Weighting and Linear Ensemble) and the feature importance-based advanced methods (Gating Network, Squeeze-Excitation Network, and Attention Network) for Vietnamese sentiment analysis. We do these ensemble techniques with five individual deep learning models: TextCNN, LSTM, GRU, LSTM + CNN, and GRU + CNN. Extensive experiments on two datasets show that the ensemble methods perform much better than any individual model and significantly outperform the competition’s winning solution with a large margin. Finally, we aim to publish our source codes to contribute to the current research community related to natural language processing.
C. V. Nguyen and K. H. Le—Equal contribution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. CoRR, abs/1607.04606 arXiv:1607.0460 (2016)
Chakriswaran, P., Vincent, D.R., Srinivasan, K., Sharma, V., Chang, C.Y., Reina, D.G.: Emotion AI-driven sentiment analysis: a survey, future research directions, and open issues. Appl. Sci. 9(24), 5462 (2019)
Conneau, A., Schwenk, H., Barrault, L., Lecun, Y.: Very deep convolutional networks for text classification (2016)
Contratres, F.G., Alves-Souza, S.N., Filgueiras, L.V.L., DeSouza, L.S.: Sentiment analysis of social network data for cold-start relief in recommender systems. In: Rocha, Á., Adeli, H., Reis, L.P., Costanzo, S. (eds.) WorldCIST’18 2018. AISC, vol. 746, pp. 122–132. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77712-2_12
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding (2019)
ElJundi, O., Antoun, W., El Droubi, N., Hajj, H., El-Hajj, W., Shaban, K.: hulmona: The universal language model in Arabic. In: Proceedings of the Fourth Arabic Natural Language Processing Workshop, pp. 68–77 (2019)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. CoRR, abs/1709.01507 arXiv:1709.01507 (2017)
Joachims, T.: A support vector method for multivariate performance measures. In: Proceedings of the 22nd International Conference on Machine Learning, ICML 2005, pp. 377–384. Association for Computing Machinery (2005)
Kim, Y.: Convolutional neural networks for sentence classification. CoRR, abs/1408.5882 arXiv:1408.5882 (2014)
Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: Bonet, B., Koenig, S. (eds.) AAAI, vol. 333, pp. 2267–2273 (2015)
Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. CoRR, abs/1508.04025 arXiv:1508.04025 (2015)
Mai, L., Le, B.: Aspect-based sentiment analysis of vietnamese texts with deep learning. In: Nguyen, N.T., Hoang, D.H., Hong, T.-P., Pham, H., Trawiński, B. (eds.) ACIIDS 2018. LNCS (LNAI), vol. 10751, pp. 149–158. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75417-8_14
Medhat, W., Hassan, A., Korashy, H.: Sentiment analysis algorithms and applications: a survey. Ain Shams Eng. J. 5(4), 1093–1113 (2014)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS 2013, vol. 2, pp. 3111–3119. Curran Associates Inc. (2013)
Nguyen, C.V., Le, K.H., Tran, A.M., Nguyen, B.T.: An efficient framework for Vietnamese sentiment analysis. In: Proceedings of The 18th International Conference on Intelligent Software Methodologies, Tools, and Techniques (SoMeT) (2020)
Nguyen, H.Q., Nguyen, Q.U.: An ensemble of shallow and deep learning algorithms for Vietnamese sentiment analysis, pp. 165–170 (2018)
Nguyen-Thanh, T., Tran, G.T.: Vietnamese sentiment analysis for hotel review based on overfitting training and ensemble learning (2019)
Nguyen-Thi, B.-T., Duong, H.-T.: A Vietnamese sentiment analysis system based on multiple classifiers with enhancing lexicon features. In: Duong, T.Q., Vo, N.-S., Nguyen, L.K., Vien, Q.-T., Nguyen, V.-D. (eds.) INISCOM 2019. LNICST, vol. 293, pp. 240–249. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30149-1_20
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, pp. 8024–8035 (2019)
Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Phan, D.H., Cao, T.D.: Applying skip-gram word estimation and SVM-based classification for opinion mining Vietnamese food places text reviews. In: Proceedings of the Fifth Symposium on Information and Communication Technology, SoICT 2014, pp. 232–239. Association for Computing Machinery (2014)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners (2019)
Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432 (2015)
Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422–1432. Association for Computational Linguistics (2015)
Trinh, S., Nguyen, L., Vo, M., Do, P.: Lexicon-based sentiment analysis of Facebook comments in Vietnamese language. In: Król, D., Madeyski, L., Nguyen, N.T. (eds.) Recent Developments in Intelligent Information and Database Systems. SCI, vol. 642, pp. 263–276. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31277-4_23
Vaswani, A., et al.: Attention is all you need. CoRR, abs/1706.03762 arXiv:1706.03762 (2017)
Vo, H.T., Lam, H.C., Nguyen, D.D., Tuong, N.H.: Topic classification and sentiment analysis for Vietnamese education survey system. Asian J. Comput. Sci. Inf. Technol. 6, 27–34 (2016)
Wang, G., Sun, J., Ma, J., Kaiquan, X., Jibao, G.: Sentiment classification: the contribution of ensemble learning. Decis. Support Syst. 57, 77–93 (2014)
Xie, Q., Dai, Z., Hovy, E., Luong, M.T., Le, Q.V.: Unsupervised data augmentation for consistency training (2019)
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen, C.V., Le, K.H., Nguyen, B.T. (2021). A Novel Approach for Enhancing Vietnamese Sentiment Classification. In: Fujita, H., Selamat, A., Lin, J.CW., Ali, M. (eds) Advances and Trends in Artificial Intelligence. From Theory to Practice. IEA/AIE 2021. Lecture Notes in Computer Science(), vol 12799. Springer, Cham. https://doi.org/10.1007/978-3-030-79463-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-79463-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-79462-0
Online ISBN: 978-3-030-79463-7
eBook Packages: Computer ScienceComputer Science (R0)