Advertisement

Cognitive Computation

, Volume 10, Issue 4, pp 639–650 | Cite as

Sentic LSTM: a Hybrid Network for Targeted Aspect-Based Sentiment Analysis

  • Yukun Ma
  • Haiyun Peng
  • Tahir Khan
  • Erik Cambria
  • Amir Hussain
Article

Abstract

Sentiment analysis has emerged as one of the most popular natural language processing (NLP) tasks in recent years. A classic setting of the task mainly involves classifying the overall sentiment polarity of the inputs. However, it is based on the assumption that the sentiment expressed in a sentence is unified and consistent, which does not hold in the reality. As a fine-grained alternative of the task, analyzing the sentiment towards a specific target and aspect has drawn much attention from the community for its more practical assumption that sentiment is dependent on a particular set of aspects and entities. Recently, deep neural models have achieved great successes on sentiment analysis. As a functional simulation of the behavior of human brains and one of the most successful deep neural models for sequential data, long short-term memory (LSTM) networks are excellent in learning implicit knowledge from data. However, it is impossible for LSTM to acquire explicit knowledge such as commonsense facts from the training data for accomplishing their specific tasks. On the other hand, emerging knowledge bases have brought a variety of knowledge resources to our attention, and it has been acknowledged that incorporating the background knowledge is an important add-on for many NLP tasks. In this paper, we propose a knowledge-rich solution to targeted aspect-based sentiment analysis with a specific focus on leveraging commonsense knowledge in the deep neural sequential model. To explicitly model the inference of the dependent sentiment, we augment the LSTM with a stacked attention mechanism consisting of attention models for the target level and sentence level, respectively. In order to explicitly integrate the explicit knowledge with implicit knowledge, we propose an extension of LSTM, termed Sentic LSTM. The extended LSTM cell includes a separate output gate that interpolates the token-level memory and the concept-level input. In addition, we propose an extension of Sentic LSTM by creating a hybrid of the LSTM and a recurrent additive network that simulates sentic patterns. In this paper, we are mainly concerned with a joint task combining the target-dependent aspect detection and targeted aspect-based polarity classification. The performance of proposed methods on this joint task is evaluated on two benchmark datasets. The experiment shows that the combination of proposed attention architecture and knowledge-embedded LSTM could outperform state-of-the-art methods in two targeted aspect sentiment tasks. We present a knowledge-rich solution for the task of targeted aspect-based sentiment analysis. Our model can effectively incorporate the commonsense knowledge into the deep neural network and be trained in an end-to-end manner. We show that the two-step attentive neural architecture as well as the proposed Sentic LSTM and H-Sentic-LSTM can achieve an improved performance on resolving the aspect categories and sentiment polarity for a targeted entity in its context over state-of-the-art systems.

Keywords

Sentiment analysis Commonsense knowledge Attention model Neural network 

Notes

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was not required as no human or animals were involved.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by any of the authors.

References

  1. 1.
    Cambria E, Das D, Bandyopadhyay S, Feraco A. A practical guide to sentiment analysis. Cham: Springer; 2017.CrossRefGoogle Scholar
  2. 2.
    Poria S, Cambria E, Bajpai R, Hussain A. A review of affective computing: from unimodal analysis to multimodal fusion. Inf Fusion 2017;37:98–125.CrossRefGoogle Scholar
  3. 3.
    Poria S, Cambria E, Hazarika D, Mazumder N, Zadeh A, Morency L.-P. Context-dependent sentiment analysis in user-generated videos. ACL; 2017. p. 873–83.Google Scholar
  4. 4.
    Chaturvedi I, Ragusa E, Gastaldo P, Zunino R, Cambria E. Bayesian network based extreme learning machine for subjectivity detection. J. Frankl. Inst 2018;355(4):1780–1797.CrossRefGoogle Scholar
  5. 5.
    Das S R, Chen MY. Yahoo! for amazon: sentiment extraction from small talk on the web. Manag Sci 2007; 53(9):1375–88.CrossRefGoogle Scholar
  6. 6.
    Morinaga S, Yamanishi K, Tateishi K, Fukushima T. Mining product reputations on the web. Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining. New York: ACM; 2002. p. 341–9.Google Scholar
  7. 7.
    Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S. Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 27–35.Google Scholar
  8. 8.
    Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, Hoste V, Apidianaki M, Tannier X, Loukachevitch N, Kotelnikov E, Bel N, Jiménez-Zafra S M, Eryiğit G. Semeval-2016 task 5: aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016). San Diego: Association for Computational Linguistics; 2016. p. 19–30.Google Scholar
  9. 9.
    Poria S, Cambria E, Gelbukh A. Aspect extraction for opinion mining with a deep convolutional neural network. Knowl-Based Syst 2016;108:42–9.CrossRefGoogle Scholar
  10. 10.
    Xia Y, Cambria E, Hussain A. Aspnet: aspect extraction by bootstrapping generalization and propagation using an aspect network. Cogn Comput 2015;7(2):241–53.CrossRefGoogle Scholar
  11. 11.
    Poria S, Chaturvedi I, Cambria E, Bisio F. Sentic LDA: improving on LDA with semantic similarity for aspect-based sentiment analysis. IJCNN; 2016. p. 4465–73.Google Scholar
  12. 12.
    Tang D, Qin B, Feng X, Liu T. Effective LSTMs for target-dependent sentiment classification. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka; 2016. p. 3298–307.Google Scholar
  13. 13.
    Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K. Adaptive recursive neural network for target-dependent twitter sentiment classification. Proceedings of the 52nd annual meeting of the association for computational linguistics (Volume 2: Short Papers). Baltimore: Association for Computational Linguistics; 2014. p. 49–54.Google Scholar
  14. 14.
    Wang B, Liakata M, Zubiaga A, Tdparse R. Procter. Multi-target-specific sentiment recognition on twitter. Proceedings of the 15th conference of the European chapter of the association for computational linguistics: volume 1, Long Papers. Valencia: Association for Computational Linguistics; 2017. p. 483–93.Google Scholar
  15. 15.
    Saeidi M, Bouchard G, Liakata M, Riedel S. Sentihood: targeted aspect based sentiment analysis dataset for urban neighbourhoods. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka: The COLING 2016 Organizing Committee; 2016. p. 1546–56.Google Scholar
  16. 16.
    Nguyen T H, Shirai K. Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. Proceedings of the 2015 conference on empirical methods in natural language processing. Lisbon: Association for Computational Linguistics; 2015. p. 2509–14.Google Scholar
  17. 17.
    Wang Y, Huang M, Zhu X, Zhao L. Attention-based LSTM for aspect-level sentiment classification. Proceedings of the 2016 conference on empirical methods in natural language processing. Austin: Association for Computational Linguistics; 2016. p. 606–15.Google Scholar
  18. 18.
    Tang D, Qin B, Liu T, Aspect level sentiment classification with deep memory network. Proceedings of the 2016 conference on empirical methods in natural language processing. Austin: Association for Computational Linguistics; 2016. p. 214–24.Google Scholar
  19. 19.
    Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 1997;9(8):1735–80.CrossRefPubMedGoogle Scholar
  20. 20.
    Cambria E, Hussain A, Havasi C, Eckl C. Common sense computing: from the society of mind to digital intuition and beyond. In: Fierrez J, Ortega J, Esposito A, Drygajlo A, Faundez-Zanuy M, editors. Biometric ID management and multimodal communication, volume 5707 of lecture notes in computer science. Berlin: Springer; 2009, pp. 252–9.Google Scholar
  21. 21.
    Baccianella S, Esuli A, Sebastiani F. Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. LREC. Valletta: European Language Resources Association (ELRA); 2010. p. 2200–2204.Google Scholar
  22. 22.
    Cambria E, Poria S, Bajpai R, Schuller B. SenticNet 4: a semantic resource for sentiment analysis based on conceptual primitives. Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers. Osaka: The COLING 2016 Organizing Committee; 2016. p. 2666– 2677.Google Scholar
  23. 23.
    Ratinov L, Roth D. Design challenges and misconceptions in named entity recognition. Proceedings of the thirteenth conference on computational natural language learning. Association for Computational Linguistics; 2009. p. 147–155.Google Scholar
  24. 24.
    Ma Y, Kim J-J, Bigot B, Khan TM. Feature-enriched word embeddings for named entity recognition in open-domain conversations. 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE; 2016. p. 6055–6059.Google Scholar
  25. 25.
    Xu Z, Liu B, Wang B, Sun C, Wang X. Incorporating loose-structured knowledge into LSTM with recall gate for conversation modeling. arXiv:1605.05110. 2016.
  26. 26.
    Li Y, Pan Q, Yang T, Wang S H, Tang J L, Cambria E. Learning word representations for sentiment analysis. Cogn Comput 2017 ;9(6):843–51.CrossRefGoogle Scholar
  27. 27.
    Ofek N, Poria S, Rokach L, Cambria E, Hussain A, Shabtai A. Unsupervised commonsense knowledge enrichment for domain-specific sentiment analysis. Cogn Comput 2016;8(3):467–77.CrossRefGoogle Scholar
  28. 28.
    Yang B, Mitchell T. Leveraging knowledge bases in LSTMs for improving machine reading. Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers). Vancouver: Association for Computational Linguistics; 2017. p. 1436–1446.Google Scholar
  29. 29.
    Cambria E, Hussain A. Sentic album: content-, concept-, and context-based online personal photo management system. Cogn Comput 2012;4(4):477–96.CrossRefGoogle Scholar
  30. 30.
    Wang Q-F, Cambria E, Liu C-L, Hussain A. Common sense knowledge for handwritten Chinese text recognition. Cogn Comput 2013;5(2):234–42.CrossRefGoogle Scholar
  31. 31.
    Cambria E, Fu J, Bisio F, Poria S. AffectiveSpace 2: enabling affective intuition for concept-level sentiment analysis. AAAI; 2015. p. 508–514.Google Scholar
  32. 32.
    Wagner J, Arora P, Cortes S, Barman U, Bogdanova D, Foster J, Dcu L. Tounsi. Aspect-based polarity classification for SemEval task 4. Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 223–229.Google Scholar
  33. 33.
    Kiritchenko S, Zhu X, Cherry C, Mohammad S. NRC-Canada-2014: detecting aspects and sentiment in customer reviews. Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Dublin: Association for Computational Linguistics and Dublin City University; 2014. p. 437–442.Google Scholar
  34. 34.
    Lakkaraju H, Socher R, Manning C. Aspect specific sentiment analysis using hierarchical deep learning. NIPS workshop on deep learning and representation learning. Curran Associates Inc.; 2014.Google Scholar
  35. 35.
    Chen P, Sun Z, Bing L, Yang W. Recurrent attention network on memory for aspect sentiment analysis. Proceedings of the 2017 conference on empirical methods in natural language processing. Copenhagen: Association for Computational Linguistics; 2017. p. 463–472.Google Scholar
  36. 36.
    Rahman A, Ng V. Conference resolution with world knowledge. Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-volume 1. Association for Computational Linguistics; 2011. p. 814–824.Google Scholar
  37. 37.
    Nakashole N, Mitchell TM. A knowledge-intensive model for prepositional phrase attachment. ACL (1); 2015. p. 365–375.Google Scholar
  38. 38.
    Ahn S, Choi H, Pärnamaa T., Bengio Y. A neural knowledge language model. arXiv:1608.00318. 2016.
  39. 39.
    Schuster M, Paliwal KK. Bidirectional recurrent neural networks. IEEE Trans Signal Process 1997;45(11): 2673–81.CrossRefGoogle Scholar
  40. 40.
    Oneto L, Bisio F, Cambria E, Anguita D. Semi-supervised learning for affective common-sense reasoning. Cogn Comput 2017;9(1):18–42.CrossRefGoogle Scholar
  41. 41.
    Lee K, Levy O, Zettlemoyer L. Recurrent additive networks. arXiv:1705.07393. 2017.
  42. 42.
    Poria S, Cambria E, Winterstein G, Huang G-B. Sentic patterns Dependency-based rules for concept-level sentiment analysis. Knowl-Based Syst 2014;69:45–63.CrossRefGoogle Scholar
  43. 43.
    Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I. SemEval-2015 task 12: aspect based sentiment analysis. Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015). Denver: Association for Computational Linguistics; 2015. p. 486–495.Google Scholar
  44. 44.
    Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems; 2013. p. 3111–3119.Google Scholar
  45. 45.
    He R, McAuley J. Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. Proceedings of the 25th international conference on world wide web. International World Wide Web Conferences Steering Committee; 2016. p. 507–517.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringNanyang Technological UniversitySingaporeSingapore
  2. 2.Alef Education ConsultancyAbu DhabiUnited Arab Emirates
  3. 3.School of Natural SciencesUniversity of StirlingStirlingUK

Personalised recommendations