Sentic LSTM: a Hybrid Network for Targeted Aspect-Based Sentiment Analysis
Sentiment analysis has emerged as one of the most popular natural language processing (NLP) tasks in recent years. A classic setting of the task mainly involves classifying the overall sentiment polarity of the inputs. However, it is based on the assumption that the sentiment expressed in a sentence is unified and consistent, which does not hold in the reality. As a fine-grained alternative of the task, analyzing the sentiment towards a specific target and aspect has drawn much attention from the community for its more practical assumption that sentiment is dependent on a particular set of aspects and entities. Recently, deep neural models have achieved great successes on sentiment analysis. As a functional simulation of the behavior of human brains and one of the most successful deep neural models for sequential data, long short-term memory (LSTM) networks are excellent in learning implicit knowledge from data. However, it is impossible for LSTM to acquire explicit knowledge such as commonsense facts from the training data for accomplishing their specific tasks. On the other hand, emerging knowledge bases have brought a variety of knowledge resources to our attention, and it has been acknowledged that incorporating the background knowledge is an important add-on for many NLP tasks. In this paper, we propose a knowledge-rich solution to targeted aspect-based sentiment analysis with a specific focus on leveraging commonsense knowledge in the deep neural sequential model. To explicitly model the inference of the dependent sentiment, we augment the LSTM with a stacked attention mechanism consisting of attention models for the target level and sentence level, respectively. In order to explicitly integrate the explicit knowledge with implicit knowledge, we propose an extension of LSTM, termed Sentic LSTM. The extended LSTM cell includes a separate output gate that interpolates the token-level memory and the concept-level input. In addition, we propose an extension of Sentic LSTM by creating a hybrid of the LSTM and a recurrent additive network that simulates sentic patterns. In this paper, we are mainly concerned with a joint task combining the target-dependent aspect detection and targeted aspect-based polarity classification. The performance of proposed methods on this joint task is evaluated on two benchmark datasets. The experiment shows that the combination of proposed attention architecture and knowledge-embedded LSTM could outperform state-of-the-art methods in two targeted aspect sentiment tasks. We present a knowledge-rich solution for the task of targeted aspect-based sentiment analysis. Our model can effectively incorporate the commonsense knowledge into the deep neural network and be trained in an end-to-end manner. We show that the two-step attentive neural architecture as well as the proposed Sentic LSTM and H-Sentic-LSTM can achieve an improved performance on resolving the aspect categories and sentiment polarity for a targeted entity in its context over state-of-the-art systems.
KeywordsSentiment analysis Commonsense knowledge Attention model Neural network
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they have no conflict of interest.
Informed consent was not required as no human or animals were involved.
Human and Animal Rights
This article does not contain any studies with human or animal subjects performed by any of the authors.
- 3.Poria S, Cambria E, Hazarika D, Mazumder N, Zadeh A, Morency L.-P. Context-dependent sentiment analysis in user-generated videos. ACL; 2017. p. 873–83.Google Scholar
- 6.Morinaga S, Yamanishi K, Tateishi K, Fukushima T. Mining product reputations on the web. Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining. New York: ACM; 2002. p. 341–9.Google Scholar
- 7.Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S. Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 27–35.Google Scholar
- 8.Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, Hoste V, Apidianaki M, Tannier X, Loukachevitch N, Kotelnikov E, Bel N, Jiménez-Zafra S M, Eryiğit G. Semeval-2016 task 5: aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). San Diego: Association for Computational Linguistics; 2016. p. 19–30.Google Scholar
- 11.Poria S, Chaturvedi I, Cambria E, Bisio F. Sentic LDA: improving on LDA with semantic similarity for aspect-based sentiment analysis. IJCNN; 2016. p. 4465–73.Google Scholar
- 12.Tang D, Qin B, Feng X, Liu T. Effective LSTMs for target-dependent sentiment classification. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka; 2016. p. 3298–307.Google Scholar
- 13.Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K. Adaptive recursive neural network for target-dependent twitter sentiment classification. Proceedings of the 52nd annual meeting of the association for computational linguistics (Volume 2: Short Papers). Baltimore: Association for Computational Linguistics; 2014. p. 49–54.Google Scholar
- 14.Wang B, Liakata M, Zubiaga A, Tdparse R. Procter. Multi-target-specific sentiment recognition on twitter. Proceedings of the 15th conference of the European chapter of the association for computational linguistics: volume 1, Long Papers. Valencia: Association for Computational Linguistics; 2017. p. 483–93.Google Scholar
- 15.Saeidi M, Bouchard G, Liakata M, Riedel S. Sentihood: targeted aspect based sentiment analysis dataset for urban neighbourhoods. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka: The COLING 2016 Organizing Committee; 2016. p. 1546–56.Google Scholar
- 16.Nguyen T H, Shirai K. Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. Proceedings of the 2015 conference on empirical methods in natural language processing. Lisbon: Association for Computational Linguistics; 2015. p. 2509–14.Google Scholar
- 17.Wang Y, Huang M, Zhu X, Zhao L. Attention-based LSTM for aspect-level sentiment classification. Proceedings of the 2016 conference on empirical methods in natural language processing. Austin: Association for Computational Linguistics; 2016. p. 606–15.Google Scholar
- 18.Tang D, Qin B, Liu T, Aspect level sentiment classification with deep memory network. Proceedings of the 2016 conference on empirical methods in natural language processing. Austin: Association for Computational Linguistics; 2016. p. 214–24.Google Scholar
- 20.Cambria E, Hussain A, Havasi C, Eckl C. Common sense computing: from the society of mind to digital intuition and beyond. In: Fierrez J, Ortega J, Esposito A, Drygajlo A, Faundez-Zanuy M, editors. Biometric ID management and multimodal communication, volume 5707 of lecture notes in computer science. Berlin: Springer; 2009, pp. 252–9.Google Scholar
- 21.Baccianella S, Esuli A, Sebastiani F. Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. LREC. Valletta: European Language Resources Association (ELRA); 2010. p. 2200–2204.Google Scholar
- 22.Cambria E, Poria S, Bajpai R, Schuller B. SenticNet 4: a semantic resource for sentiment analysis based on conceptual primitives. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka: The COLING 2016 Organizing Committee; 2016. p. 2666– 2677.Google Scholar
- 23.Ratinov L, Roth D. Design challenges and misconceptions in named entity recognition. Proceedings of the thirteenth conference on computational natural language learning. Association for Computational Linguistics; 2009. p. 147–155.Google Scholar
- 24.Ma Y, Kim J-J, Bigot B, Khan TM. Feature-enriched word embeddings for named entity recognition in open-domain conversations. 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE; 2016. p. 6055–6059.Google Scholar
- 25.Xu Z, Liu B, Wang B, Sun C, Wang X. Incorporating loose-structured knowledge into LSTM with recall gate for conversation modeling. arXiv:1605.05110. 2016.
- 28.Yang B, Mitchell T. Leveraging knowledge bases in LSTMs for improving machine reading. Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers). Vancouver: Association for Computational Linguistics; 2017. p. 1436–1446.Google Scholar
- 31.Cambria E, Fu J, Bisio F, Poria S. AffectiveSpace 2: enabling affective intuition for concept-level sentiment analysis. AAAI; 2015. p. 508–514.Google Scholar
- 32.Wagner J, Arora P, Cortes S, Barman U, Bogdanova D, Foster J, Dcu L. Tounsi. Aspect-based polarity classification for SemEval task 4. Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 223–229.Google Scholar
- 33.Kiritchenko S, Zhu X, Cherry C, Mohammad S. NRC-Canada-2014: detecting aspects and sentiment in customer reviews. Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 437–442.Google Scholar
- 34.Lakkaraju H, Socher R, Manning C. Aspect specific sentiment analysis using hierarchical deep learning. NIPS workshop on deep learning and representation learning. Curran Associates Inc.; 2014.Google Scholar
- 35.Chen P, Sun Z, Bing L, Yang W. Recurrent attention network on memory for aspect sentiment analysis. Proceedings of the 2017 conference on empirical methods in natural language processing. Copenhagen: Association for Computational Linguistics; 2017. p. 463–472.Google Scholar
- 36.Rahman A, Ng V. Conference resolution with world knowledge. Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-volume 1. Association for Computational Linguistics; 2011. p. 814–824.Google Scholar
- 37.Nakashole N, Mitchell TM. A knowledge-intensive model for prepositional phrase attachment. ACL (1); 2015. p. 365–375.Google Scholar
- 38.Ahn S, Choi H, Pärnamaa T., Bengio Y. A neural knowledge language model. arXiv:1608.00318. 2016.
- 41.Lee K, Levy O, Zettlemoyer L. Recurrent additive networks. arXiv:1705.07393. 2017.
- 43.Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I. SemEval-2015 task 12: aspect based sentiment analysis. Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015). Denver: Association for Computational Linguistics; 2015. p. 486–495.Google Scholar
- 44.Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems; 2013. p. 3111–3119.Google Scholar
- 45.He R, McAuley J. Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. Proceedings of the 25th international conference on world wide web. International World Wide Web Conferences Steering Committee; 2016. p. 507–517.Google Scholar