Abstract
Sentiment analysis is a fundamental branch of natural language processing. It is an essential task of identifying and extracting sentiment in opinionated data from sources such as social media, product feedback or blogs. Deep learning-based approaches have exceeded human-level performance in areas such as computer vision and speech recognition. Deep learning is widely accepted as the most promising in machine learning. In this chapter, we survey and analyse the current trends and advances in deep learning-based sentiment analysis approaches for document-level, sentence-level and aspect-based sentiment analysis for short and long text. A detailed discussion of deep learning architectures for sentiment analysis is provided. The studied approaches are classified into coarse-grain (including document and sentence level), fine-grain (includes target and aspect level) and cross-domain. Lastly, we provide a summary and in-depth analysis of the surveyed studies, for each of the aforementioned categories. The overwhelming number of studies explored convolutional neural networks (CNNs), long short-term memory (LSTM), gated recurrent unit (GRU) and attention mechanism. For coarse-grain sentiment analysis, LSTM and CNN-based models compete on performance, but it is CNNs that offer reduced model complexity and training overhead. Fine-grain sentiment analysis requires a model to learn complex interactions between target/aspect words and opinion words. Bi-directional LSTM and attention mechanisms offer the most promise, although CNN-based models have been adept at aspect extraction. The efforts in cross-domain sentiment analysis are dominated by LSTM and attention models. Our survey of cross-domain approaches revealed the use of multitask learning, adversarial training and joint training for domain adaptation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Koncz, P., and J. Paralic. 2011. An approach to feature selection for sentiment analysis. In 2011 15th IEEE International Conference on Intelligent Engineering Systems, 357–362. IEEE.
Zhang, H., Z. Yu, M. Xu, and Y. Shi. 2011. Feature-level sentiment analysis for Chinese product reviews. In 2011 3rd International Conference on Computer Research and Development, vol. 2, 135–140. IEEE.
Hu, M., and B. Liu. 2004. Mining opinion features in customer reviews. AAAI 4 (4): 755–760.
Biere, S. 2019. Hate speech detection using natural language processing techniques (online) Beta.vu.nl. Available at: https://beta.vu.nl/nl/Images/werkstuk-biere_tcm235-893877.pdf. Accessed 3 Jun 2019.
Nahar, V., S. Unankard, X. Li, and C. Pang. 2012. Sentiment analysis for effective detection of cyber bullying. In Asia-Pacific Web Conference, 767–774. Berlin, Heidelberg: Springer.
Roman Steinberg, u. 2019. 6 areas where artificial neural networks outperform humans (online) VentureBeat. Available at: https://venturebeat.com/2017/12/08/6-areas-where-artificial-neural-networks-outperform-humans/. Accessed 10 Apr 2019.
Zhang, L., Wang, S. and Liu, B. 2018. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, e1253.
Llombart, O.R. 2017. Using machine learning techniques for sentiment analysis. Published in June of 2017.
Kursuncu, U., M. Gaur, U. Lokala, K. Thirunarayan, A. Sheth, and I.B. Arpinar. 2019. Predictive analysis on Twitter: Techniques and applications. In Emerging Research Challenges and Opportunities in Computational Social Network Analysis and Mining, 67–104. Cham: Springer.
Giachanou, A., and F. Crestani. 2016. Like it or not: A survey of twitter sentiment analysis methods. ACM Computing Surveys (CSUR) 49 (2): 28.
Mariel, W.C.F., S. Mariyah, and S. Pramana. 2018. Sentiment analysis: A comparison of deep learning neural network algorithm with SVM and naÏŠve Bayes for Indonesian text. In Journal of Physics: Conference Series, vol. 971(1), 012049. IOP Publishing.
Pradhan, V.M., J. Vala, and P. Balani. 2016. A survey on sentiment analysis algorithms for opinion mining. International Journal of Computer Applications 133 (9): 7–11.
Xia, R., C. Zong, and S. Li. 2011. Ensemble of feature sets and classification algorithms for sentiment classification. Information Sciences 181 (6): 1138–1152.
Joachims, T. 1998. Text categorization with support vector machines: learning with many relevant features. In 10th European Conference on Machine Learning (ECML’98).
Rennie, J. and R. Rifkin. 2001. Improving multiclass text classification with the support vector machine. MIT, Technical Report. AIM-2001-026.2001.
Ly, D.K., K. Sugiyama, Z. Lin, and M.Y. Kan. 2011. June. Product review summarization from a deeper perspective. In Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries, 311–314. ACM.
Agarwal, B, N. Mittal. 2016. Prominent feature extraction for sentiment analysis. In Springer Book Series: Socio-Affective Computing series, 1–115. Springer International Publishing. ISBN: 978-3-319-25343-5, https://doi.org/10.1007/978-3-319-25343-5.
Somprasertsri, G. and P. Lalitrojwong. 2008. Automatic product feature extraction from online product reviews using maximum entropy with lexical and syntactic features. In 2008 IEEE International Conference on Information Reuse and Integration, 250–255. IEEE.
Wang, B, and H. Wang. 2008. Bootstrapping both product features and opinion words from Chinese customer reviews with cross-inducing. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I.
Zhang, L., B. Liu, S.H. Lim, and E. O’Brien-Strain. 2010. Extracting and ranking product features in opinion documents. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, 1462–1470. Association for Computational Linguistics.
Sriram, B., D. Fuhry, E. Demir, H. Ferhatosmanoglu, and M. Demirbas. 2010. Short text classification in twitter to improve information filtering. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 841–842. ACM.
Cambria, E.e.a. 2013. Statistical approaches to concept-level sentiment analysis. IEEE Intelligent Systems 28 (3): 6–9.
Mokken, M.M.K.R.J, and M.D. Rijke. 2004. Using wordnet to measure semantic orientation of adjectives. In 4th International Conference on Language Resources and Evaluation, 1115–1118.
Esuli, A, and F. Sebastiani. 2005. Determining the semantic orientation of terms through gloss classification. In 14th ACM International Conference on Information and Knowledge Management, 617–624.
Peng, T.-C., and C.-C. Shih. 2010. An unsupervised snippet-based sentiment classification method for chinese unknown phrases without using reference word pairs. Journal of Computing 2 (8).
Melville, P., W. Gryc, and R.D. Lawrence. 2009. Sentiment analysis of blogs by combining lexical knowledge with text classification. In 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ser. KDD ’09 1275–1284. ACM.
Han, E. 1999. Text categorisation using weight adjusted k-nearest neighbour classification. Ph.D. dissertation, University of Minnesota.
A.P. Jain, and V.D. Katkar. 2015. Sentiments analysis of Twitter data using data mining. In International Conference on Information Processing (ICIP).
Turney, P., and M. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems 21 (4): 315–346.
Wiegand, M., A. Balahur, B. Roth, D. Klakow, and A. es Montoyo. 2010. A survey on the role of negation in sentiment analysis. In Proceedings of the Workshop on Negation and Speculation in Natural Language Processing, ser., 60–68.
Bifet, A., and E. Frank. 2010. Sentiment knowledge discovery in Twitter streaming data. In Proceedings of the 13th International Conference on Discovery Science, 15, Ed.
Cambria, E., B. Schuller, Y. Xia, and C. Havasi. 2013. New avenues in opinion mining and sentiment analysis. IEEE Intelligent Systems 28 (2): 15–21.
Pang, B., L. Lee, and S. Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, 79–86.
Ficamos, P., and Y. Liu. 2016. A topic based approach for sentiment analysis on twitter data. International Journal of Advanced Computer Science and Applications 7 (12).
Cambria, E., and A. Hussain. 2015. Sentic computing: A common-sense-based framework for concept-level sentiment analysis. Springer Publishing Company, Incorporated.
LeCun, Y., L. Bottou, Y. Bengio, and P. Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86 (11): 2278–2324.
Hochreiter, S., and J. Schmidhuber. 1997. Long short-term memory. Neural Computation 9 (8): 1735–1780.
Cho, K., B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
Chung, J., C. Gulcehre, K. Cho, and Y. Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
Bahdanau, D., K. Cho, and Y. Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
Akhtar, S., A. Kumar, A. Ekbal, and P. Bhattacharyya. 2016. A hybrid deep learning architecture for sentiment analysis. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers 482–493. Osaka, Japan: The COLING 2016 Organizing Committee.
Johnson R, and T. Zhang. 2015. Effective use of word order for text categorization with convolutional neural networks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2015).
Salinca, A. 2017. Convolutional neural networks for sentiment classification on business reviews. arXiv preprint arXiv:1710.05978.
Rani, S., and P. Kumar. 2018. Deep learning based sentiment analysis using convolution neural network. Arabian Journal for Science and Engineering.
Xu, J., D. Chen, X. Qiu, and X. Huang. 2016. Cached long short-term memory neural networks for document-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.
Wang, Y., A. Sun, J. Han, Y. Liu, and X. Zhu. 2018. Sentiment analysis by capsules. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, 1165–1174. International World Wide Web Conferences Steering Committee.
Zadeh, A., M. Chen, S. Poria, E. Cambria, and L.-P. Morency. 2017. Tensor fusion network for multimodal sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.
Nguyen, D., K. Vo, D. Pham, M. Nguyen, and T. Quan. 2017. A deep architecture for sentiment analysis of news articles. In International Conference on Computer Science, Applied Mathematics and Applications, 129–140. Cham: Springer.
Liu, P., X. Qiu, and X. Huang. 2016. Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101.
Zhou X, X. Wan, J. Xiao. 2016. Attention-based LSTM network for cross-lingual sentiment classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2016).
Yang, Z., D. Yang, C. Dyer, X. He, A. Smola, and E. Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1480–1489.
Baziotis, C., N. Pelekis, and C. Doulkeridis. 2017. Datastories at semeval-2017 task 4: Deep lstm with attention for message-level and topic-based sentiment analysis. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 747–754.
Poria, S., I. Chaturvedi, E. Cambria, and A. Hussain. 2016. Convolutional MKL based multimodal emotion recognition and sentiment analysis. In 2016 IEEE 16th International Conference on Data Mining (ICDM), 439–448. IEEE.
Wang, B. 2018. Disconnected recurrent neural networks for text categorization. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long Papers, 2311–2320.
Chen, T., R. Xu, Y. He, and X. Wang. 2017. Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN. Expert Systems with Applications 72: 221–230.
Cliche, M. 2017. BB_twtr at SemEval-2017 Task 4: Twitter sentiment analysis with CNNs and LSTMs. arXiv preprint arXiv:1704.06125.
Yin Y, Y. Song, M. Zhang. 2017. Document-level multi-aspect sentiment classification as machine comprehension. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2017).
Xue, W, and T. Li. 2018. Aspect based sentiment analysis with gated convolutional networks. arXiv preprint arXiv:1805.07043.
Zhu, P., and T. Qian. 2018. Enhanced aspect level sentiment classification with auxiliary memory. In Proceedings of the 27th International Conference on Computational Linguistics, 1077–1087.
Wang, W., S.J. Pan, and D. Dahlmeier. 2018. Memory networks for fine-grained opinion mining. Artificial Intelligence 265: 1–17.
Huang, B., Y. Ou, and K.M. Carley. 2018. Aspect level sentiment classification with attention-over-attention neural networks. arXiv preprint arXiv:1804.06536.
Zheng, S, and R. Xia. 2018. Left-center-right separated neural network for aspect-based sentiment analysis with rotatory attention. arXiv preprint arXiv:1802.00892.
Wang, B, and W. Lu. 2018. Learning latent opinions for aspect-level sentiment classification. In Thirty-Second AAAI Conference on Artificial Intelligence.
Zhang, M., Y. Zhang, and D.T. Vo. 2016. Gated neural networks for targeted sentiment analysis. In AAAI, 3087–3093.
Poria, S., E. Cambria, and A. Gelbukh. 2016. Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems 108: 42–49.
Chen, X, and C. Cardie. 2018. Multinomial adversarial networks for multi-domain text classification. arXiv preprint arXiv:1802.05694.
Ji, J., C. Luo, X. Chen, L. Yu, and P. Li. 2018. Cross-domain sentiment classification via a bifurcated-LSTM. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 681–693. Cham: Springer.
Ruder, S, and B. Plank. 2018. Strong baselines for neural semi-supervised learning under domain shift. arXiv preprint arXiv:1804.09530.
Li, Z, Y. Zhang, Y. Wei, Y. Wu, and Q. Yang. 2017. End-to-end adversarial memory network for cross-domain sentiment classification. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2017).
Li, Z., Y. Wei, Y. Zhang, and Q. Yang. 2018. Hierarchical attention transfer network for cross-domain sentiment classification. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, AAAI 2018, New Orleans, Lousiana, USA, 2–7 Feb 2018.
Nam, H, and B. Han. 2016. Learning multi-domain convolutional neural networks for visual tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4293–4302.
Baevski, A., Edunov, S., Liu, Y., Zettlemoyer, L. and Auli, M., 2019. Cloze-driven Pretraining of Self-attention Networks. arXiv preprint arXiv:1903.07785.
Johnson, R. and Zhang, T. 2017. Deep pyramid convolutional neural networks for text categorization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long Papers, 562–570.
dos Santos, C. 2014. Think positive: Towards Twitter sentiment analysis from scratch. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), 647–651.
Deriu, J., M. Gonzenbach, F. Uzdilli, A. Lucchi, V.D. Luca, and M. Jaggi. 2016. Swisscheese at semeval-2016 task 4: Sentiment classification using an ensemble of convolutional neural networks with distant supervision. In Proceedings of the 10th International Workshop on Semantic Evaluation (No. CONF, 1124–1128).
Rouvier, M., and B. Favre. 2016. SENSEI-LIF at SemEval-2016 task 4: Polarity embedding fusion for robust sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), 202–208.
Gray, S., A. Radford, and D.P. Kingma. 2017. Gpu kernels for block-sparse weights. arXiv preprint arXiv:1711.09224.
Howard, J., and S. Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146.
Johnson, R., and T. Zhang. 2016. Supervised and semi-supervised text categorization using LSTM for region embeddings. arXiv preprint arXiv:1602.02373.
Baziotis, C., N. Pelekis, and C. Doulkeridis. 2017. Datastories at semeval-2017 task 4: Deep lstm with attention for message-level and topic-based sentiment analysis. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 747–754.
Yu, Z., and G. Liu. 2019. Sliced recurrent neural networks (online) arXiv.org. Available at: https://arxiv.org/abs/1807.02291. Accessed 11 Apr 2019.
Xu, S., H. Liang, and T. Baldwin. 2016. Unimelb at semeval-2016 tasks 4a and 4b: An ensemble of neural networks and a word2vec based model for sentiment classification. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), 183–189.
Rouvier, M. 2017. LIA at SemEval-2017 task 4: An ensemble of neural networks for sentiment classification. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 760–765.
Liu, X., P. He, W. Chen, and J. Gao. 2019. Multi-task deep neural networks for natural language understanding. arXiv preprint arXiv:1901.11504.
Tang, D., F. Wei, B. Qin, T. Liu, and M. Zhou. 2014. Coooolll: A deep learning system for twitter sentiment classification. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), 208–212.
Kar, S., S. Maharjan, and T. Solorio. 2017. RiTUAL-UH at SemEval-2017 task 5: Sentiment analysis on financial data using neural networks. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 877–882.
Khalil, T., and S.R. El-Beltagy. 2016. Niletmrg at semeval-2016 task 5: Deep convolutional neural networks for aspect category and sentiment extraction. In Proceedings of the 10th International Workshop on Semantic Evaluation (SEMEVAL-2016), 271–276.
Wang, W., S.J. Pan, D. Dahlmeier, and X. Xiao. 2016. Recursive neural conditional random fields for aspect-based sentiment analysis. arXiv preprint arXiv:1603.06679.
Socher, R., A. Perelygin, J. Wu, J. Chuang, C.D. Manning, A. Ng, and C. Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 1631–1642.
Liu, F., T. Cohn, and T. Baldwin. 2018. Recurrent entity networks with delayed memory update for targeted aspect-based sentiment analysis. arXiv preprint arXiv:1804.11019.
Patro, B.N., V.K. Kurmi, S. Kumar, and V.P. Namboodiri. 2018. Learning semantic sentence embeddings using pair-wise discriminator. arXiv preprint arXiv:1806.00807.
Brahma, S. 2019. Improved sentence modeling using suffix bidirectional LSTM. (online) Export.arxiv.org. Available at: http://export.arxiv.org/abs/1805.07340. Accessed 11 Apr 2019.
Peters, M.E., M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365.
Saeidi, M., G. Bouchard, M. Liakata, and S. Riedel. 2016. Sentihood: Targeted aspect based sentiment analysis dataset for urban neighbourhoods. arXiv preprint arXiv:1610.03771.
Tang, D., B. Qin, and T. Liu. 2016. Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900.
He, R., W.S. Lee, H.T. Ng, and D. Dahlmeier. 2018. Effective attention modeling for aspect-level sentiment classification. In Proceedings of the 27th International Conference on Computational Linguistics, 1121–1131.
Wang, Y., M. Huang, and L. Zhao. 2016. Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 606–615.
He, R., W.S. Lee, H.T. Ng, and D. Dahlmeier. 2018. Exploiting document knowledge for aspect-level sentiment classification. arXiv preprint arXiv:1806.04346.
Ma, D., S. Li, X. Zhang, and H. Wang. 2017. Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893.
Ma, Y., H. Peng, and E. Cambria. 2018. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In Thirty-Second AAAI Conference on Artificial Intelligence.
Li, Z., Y. Wei, Y. Zhang, X. Zhang, X. Li, and Q. Yang, Q. 2018. Exploiting coarse-to-fine task transfer for aspect-level sentiment classification. arXiv preprint arXiv:1811.10999.
Li, L., Y. Liu, and A. Zhou. 2018. Hierarchical attention based position-aware network for aspect-level sentiment analysis. In Proceedings of the 22nd Conference on Computational Natural Language Learning, 181–189.
Moro, G., A. Pagliarani, R. Pasolini, and C. Sartori. 2018. Cross-domain & in-domain sentiment analysis with memory-based deep neural networks. In Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management.
Aggarwal, U., and G. Aggarwal. 2017. Sentiment Analysis: A Survey. International Journal of Computer Sciences and Engineering 5 (5): 222–225.
Chen, P., Z. Sun, L. Bing, and W. Yang. 2017. Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 452–461.
Liu, Q., Y. Zhang, and J. Liu. 2018. Learning domain representation for multi-domain sentiment classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, (Long Papers), vol. 1, 541–550.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Ahmet, A., Abdullah, T. (2020). Recent Trends and Advances in Deep Learning-Based Sentiment Analysis. In: Agarwal, B., Nayak, R., Mittal, N., Patnaik, S. (eds) Deep Learning-Based Approaches for Sentiment Analysis. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-15-1216-2_2
Download citation
DOI: https://doi.org/10.1007/978-981-15-1216-2_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1215-5
Online ISBN: 978-981-15-1216-2
eBook Packages: EngineeringEngineering (R0)