Skip to main content
SpringerLink
Log in

Sentiment knowledge-induced neural network for aspect-level sentiment analysis

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Aspect-based sentiment analysis has been a popular topic in natural language processing in recent years that aims to determine the sentiment polarity of a specific aspect in one context. However, most existing models only focus on feature extraction and ignore the significant role of words with sentiment tendency (e.g. good, terrible), which results in low classification accuracy. In this paper, a sentiment knowledge-based bidirectional encoder representation from transformers (SK-BERT) is proposed to overcome this shortcoming. To introduce sentiment knowledge, SK-BERT first integrates sentiment knowledge words into independent sequences and then encodes the sequence and context into static and dynamic vectors with the BERT pretrained models, respectively. All vectors are sent to the sentiment centre to generate different dimension representations for classification. We evaluate our model on three widely used datasets. Experimental results show that the proposed SK-BERT model outperforms other state-of-the-art models. Furthermore, visualization experiments are implemented to prove the rationality of SK-BERT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability Statement

Datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. https://www.cs.uic.edu/liub/FBS/sentiment-analysis.html.

References

  1. Ibrahim M, Bajwa IS, Ul-Amin R, Kasi B (2019) A neural network-inspired approach for improved and true movie recommendations. Comput Intell Neurosci 2019:4589060

    Article  Google Scholar 

  2. Farimani SA, Jahan MV, Fard AM, Tabbakh SRK (2022) Investigating the informativeness of technical indicators and news sentiment in financial market price prediction. Knowl-Based Syst 247:108742

    Article  Google Scholar 

  3. Li W, Shao W, Ji S, Cambria E (2022) BiERU: bidirectional emotional recurrent unit for conversational sentiment analysis. Neurocomputing 467:73–82

    Article  Google Scholar 

  4. Gong C, Yu J, Xia R (2020) Unified feature and instance based domain adaptation for aspect-based sentiment analysis. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) pp 7035–7045

  5. Ali W, Yang Y, Qiu X, Ke Y, Wang Y (2021) Aspect-level sentiment analysis based on bidirectional-GRU in SIoT. IEEE Access 9:69938–69950

    Article  Google Scholar 

  6. Deng Y, Lei H, Li X, Lin Y, Cheng W, Yang S (2021) Attention capsule network for aspect-level sentiment classification. KSII Trans Intern Inf Syst (TIIS) 15(4):1275–1292

    Google Scholar 

  7. Balaganesh N, Muneeswaran K (2021) A novel aspect-based sentiment classifier using whale optimized adaptive neural network. Neural Comput Appl 34(5):4003–4012

    Article  Google Scholar 

  8. Chen C, Teng Z, Zhang Y (2020) Inducing target-specific latent structures for aspect sentiment classification. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) pp 5596–5607

  9. Long W, Tang YR, Tian YJ (2018) Investor sentiment identification based on the universum SVM. Neural Comput Appl 30(2):661–670

    Article  Google Scholar 

  10. Awwalu J, Bakar AA, Yaakub MR (2019) Hybrid N-gram model using Naive Bayes for classification of political sentiments on Twitter. Neural Comput Appl 31(12):9207–9220

    Article  Google Scholar 

  11. Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B (2016) Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics 2: 207–212

  12. Chen J, Yan S, Wong KC (2020) Verbal aggression detection on Twitter comments: convolutional neural network for short-text sentiment analysis. Neural Comput Appl 32(15):10809–10818

    Article  Google Scholar 

  13. Kipf T N, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv:1609.02907

  14. Tang D, Qin B, Feng X, Liu T (2015) Effective LSTMs for target-dependent sentiment classification. arXiv:1512.01100

  15. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing pp 606–615

  16. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. arXiv:1805.07043

  17. Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. arXiv:1805.01086

  18. Zhang C, Li Q, Song D (2019) Aspect-based sentiment classification with aspect-specific graph convolutional networks. arXiv:1909.03477

  19. Zhang M, Qian T (2020) Convolution over hierarchical syntactic and lexical graphs for aspect level sentiment analysis. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) pp 3540–3549

  20. Liang B, Su H, Gui L, Cambria E, Xu R (2022) Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks. Knowl-Based Syst 235:107643

    Article  Google Scholar 

  21. Devlin J, Chang M W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

  22. Xu H, Liu B, Shu L, Yu P S (2019) BERT post-training for review reading comprehension and aspect-based sentiment analysis. arXiv:1904.02232

  23. Song Y, Wang J, Jiang T, Liu Z, Rao Y (2019) Attentional encoder network for targeted sentiment classification. arXiv:1902.09314

  24. Cambria E, Liu Q, Decherchi S, Xing F, Kwok K (2022) SenticNet 7: a commonsense-based neurosymbolic AI framework for explainable sentiment analysis. In: Proceedings of LREC 2022

  25. Zhao A, Yu Y (2021) Knowledge-enabled BERT for aspect-based sentiment analysis. Knowl-Based Syst 227:107220

    Article  Google Scholar 

  26. Zhou J, Huang JX, Hu QV, He L (2020) Sk-gcn: modeling syntax and knowledge via graph convolutional network for aspect-level sentiment classification. Knowl-Based Syst 205:106292

    Article  Google Scholar 

  27. Shiyun C, Xin L, Yanghua X, Liang H (2019) Sentiment commonsense induced sequential neural networks for sentiment classification. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 1021–1030

  28. Chen X, Rao Y, Xie H, Wang FL, Zhao Y, Yin J (2019) Sentiment classification using negative and intensive sentiment supplement information. Data Sci Eng 4(2):109–118

    Article  Google Scholar 

  29. Huan JL, Sekh AA, Quek C, Prasad DK (2021) Emotionally charged text classification with deep learning and sentiment semantic. Neural Comput Appl 34(3):2341–2351

    Article  Google Scholar 

  30. Cambria E, Das D, Bandyopadhyay S, Feraco A (2017) Affective computing and sentiment analysis. IEEE Intell Syst 31(2):102–107

    Article  Google Scholar 

  31. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Eryiğit G (2016) Semeval-2016 task 5: aspect based sentiment analysis. In: International workshop on semantic evaluation pp 19–30

  32. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics 2:49–54

  33. Valdivia A, Luzón MV, Cambria E, Herrera F (2018) Consensus vote models for detecting and filtering neutrality in sentiment analysis. Inf Fusion 44:126–135

    Article  Google Scholar 

  34. Pennington J, Socher R, Manning C D (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) pp 1532–1543

  35. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) Nrc-canada-2014: detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014) pp 437–442

  36. Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. arXiv:1605.08900

  37. Lin P, Yang M, Lai J (2019) Deep mask memory network with semantic dependency and context moment for aspect level sentiment classification. In: IJCAI pp 5088–5094

  38. Fan F, Feng Y, Zhao D (2018) Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 conference on empirical methods in natural language processing (EMNLP) pp 3433–3442

  39. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. arXiv:1709.00893

  40. Huang B X, Ou Y L, Carley K M (2018) Aspect-level sentiment classification with attention-over-attention neural networks. In: 32nd youth academic annual conference of Chinese Association of Automation (YAC) pp 197–206

  41. Song Y, Wang J, Jiang T, Liu Z, Rao, Y (2019) Attentional encoder network for targeted sentiment classification. arXiv:1902.09314

  42. Chen Y, Kong L, Wang Y, Kong D (2021) Multi-grained attention representation with ALBERT for aspect-level sentiment classification. IEEE Access 9:106703–106713

    Article  Google Scholar 

  43. Jiang B, Hou J, Zhou W, Yang C, Wang S, Pang L (2020). Metnet: a mutual enhanced transformation network for aspect-based sentiment analysis. In: Proceedings of the 28th international conference on computational linguistics pp 162–172

  44. Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579–2605

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of electronic information, Wuhan University, Bayi Road, Wuhan, 430072, China

    Hao Yan, Benshun Yi & Huixin Li

  2. R & D center, Hunan Guotian Electronic Technology Co., Ltd, Yuelu Road, Changsha, 410006, China

    Danqing Wu

Authors
  1. Hao Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benshun Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huixin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Danqing Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benshun Yi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, H., Yi, B., Li, H. et al. Sentiment knowledge-induced neural network for aspect-level sentiment analysis. Neural Comput & Applic 34, 22275–22286 (2022). https://doi.org/10.1007/s00521-022-07698-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07698-0

Keywords

Search

Navigation