Skip to main content
SpringerLink
Log in

Dynamic multichannel fusion mechanism based on a graph attention network and BERT for aspect-based sentiment classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Aspect-based sentiment classification aims to predict the sentiment polarity on given aspect terms in a sentence. Recent works have incorporated syntactic information by developing graph neural networks (GNNs) over dependency trees to better establish connections between aspect items and their related context. However, the advancement is restricted because the dependency tree derived by the external parser is not entirely accurate, especially for high complexity and arbitrary expression datasets. To address this constraint, we propose a dynamic multichannel fusion mechanism based on the Graph AttenTion network and BERT (DMF-GAT-BERT), which regards the complementarity of semantic and syntactic information captured by GAT and BERT, respectively. Specifically, to alleviate the damage of incorrect dependency tree information to the model, we propose a two-layer dynamic fusion mechanism to adaptively adjust the fusion weight of semantic and syntax-related information channels. In addition, to capture accurate syntactic features, we propose an attentive layer ensemble (ALE) to integrate the contextual features learned by GAT in different layers. We conducted experiments on four datasets with different complexity, the Laptop, Restaurant, Twitter, and MAMS datasets, and achieved 80.38%, 86.10%, 76.22%, and 83.86% accuracy, respectively, outperforming robust baseline approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Liu B (2012) Sentiment analysis and opinion mining. Synth Lect Hum Lang Technol 5(1):1–167

    Article  Google Scholar 

  2. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the international workshop on semantic evaluations, pp 27–35

  3. Jin W, Ho H H, Srihari R K (2009) A novel lexicalized hmm-based learning framework for web opinion mining. In: Proceedings of the 26th annual international conference on machine learning, pp 465–472

  4. Wagner J, Arora P, Cortes S, Barman U, Bogdanova D, Foster J, Tounsi L (2014) Dcu: aspect-based polarity classification for semeval task 4. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 223–229

  5. Nazir A, Rao Y, Wu L, Sun L (2020) Issues and challenges of aspect-based sentiment analysis: a comprehensive survey. IEEE Trans Affect Comput. In press

  6. Tang D, Qin B, Feng X, Liu T (2010) Effective lstms for target-dependent sentiment classification. In: COLING, pp 3298–3307

  7. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th annual meeting of the association for computational linguistics, vol 2018, pp 2514–2523

  8. Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: EMNLP, pp 214–224

  9. Liu H, Chatterjee I, Zhou M, Lu X S, Abusorrah A (2020) Aspect-based sentiment analysis: a survey of deep learning methods. IEEE Trans Comput Soc Syst 7(6):1358–1375

    Article  Google Scholar 

  10. TN Kipf M W (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th international conference on learning representations

  11. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: Proceedings of the 6th international conference on learning representations

  12. Zhang C, Li Q, Song D (2019) Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: EMNLP-IJCNLP, pp 4560–4570

  13. Tian Y, Chen G, Song Y (2021) Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 2910–2922

  14. Huang L, Sun X, Li S, Zhang L, Wang H (2020) Syntax-aware graph attention network for aspect-level sentiment classification. In: Proceedings of the 28th international conference on computational linguistics, pp 799–810

  15. Wang K, Shen W, Yang Y, Quan X, Wang R (2020) Relational graph attention network for aspect-based sentiment analysis. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 3229–3238

  16. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, pp 4171–4186

  17. Wang A, Singh A, Michael J, Hill F, Levy O, Bowman S R (2019) Glue: a multi-task benchmark and analysis platform for natural language understanding. In: Proceedings of the 7th international conference on learning representations

  18. Xu H, Zhengyan Z, Ning D, Yuxian G, Xiao L, Yuqi H, Jiezhong Q, Liang Z, Wentao H, Minlie H et al (2021) Pre-trained models: past, present and future. AI Open

  19. Li R, Chen H, Feng F, Ma Z, Wang X, Hovy E (2021) Dual graph convolutional networks for aspect-based sentiment analysis. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing, pp 6319–6329

  20. Jawahar G, Sagot B, Seddah D (2019) What does bert learn about the structure of language?. In: ACL 2019-57th annual meeting of the association for computational linguistics, pp 3651–3657

  21. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606–615

  22. Tay Y, Tuan L A, Hui S C (2018) Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

  23. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. In: Proceedings of the 26th international joint conference on artificial intelligence, pp 4068–4074

  24. Fan F, Feng Y, Zhao D (2018) Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 3433–3442

  25. Song Y, Wang J, Jiang T, Liu Z, Rao Y (2019) Attentional encoder network for targeted sentiment classification. arXiv:1902.09314

  26. Huang B, Carley K M (2019) Parameterized convolutional neural networks for aspect level sentiment classification. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 1091–1096

  27. Zhang X, Gao T (2020) Multi-head attention model for aspect level sentiment analysis. J Intell Fuzzy Syst 38(1):89–96

    Article  Google Scholar 

  28. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. Computer Science

  29. Pennington J, Socher R, Manning C D (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  30. Pota M, Ventura M, Fujita H, Esposito M (2021) Multilingual evaluation of pre-processing for bert-based sentiment analysis of tweets. Expert Syst Appl 181:115119

    Article  Google Scholar 

  31. Xu H, Liu B, Shu L, Yu P S (2019) Bert post-training for review reading comprehension and aspect-based sentiment analysis. In: Proceedings of NAACL-HLT, pp 2324–2335

  32. Sun C, Huang L, Qiu X (2019) Utilizing bert for aspect-based sentiment analysis via constructing auxiliary sentence. In: Proceedings of NAACL-HLT, pp 380–385

  33. Gao Z, Feng A, Song X, Wu X (2019) Target-dependent sentiment classification with bert. IEEE Access 7:154290–154299

    Article  Google Scholar 

  34. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, pp 49–54

  35. Nguyen T H, Shirai K (2015) Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 2509–2514

  36. Phan M H, Ogunbona P O (2020) Modelling context and syntactical features for aspect-based sentiment analysis. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 3211–3220

  37. Liang Y, Meng F, Zhang J, Chen Y, Xu J, Zhou J (2021) A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis. Neurocomputing 454:291–302

    Article  Google Scholar 

  38. Xu K, Zhao H, Liu T (2020) Aspect-specific heterogeneous graph convolutional network for aspect-based sentiment classification. IEEE Access 8:139346–139355

    Article  Google Scholar 

  39. Huang B, Carley K M (2019) Syntax-aware aspect level sentiment classification with graph attention networks. In: Proceedings of EMNLP-IJCNLP, pp 5472–5480

  40. Liang B, Su H, Gui L, Cambria E, Xu R (2021) Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks. Knowl-Based Syst, 107643

  41. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, pp 49–54

  42. Jiang Q, Chen L, Xu R, Ao X, Yang M (2019) A challenge dataset and effective models for aspect-based sentiment analysis. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 6280–6285

  43. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 249–256

  44. Kingma D P, Ba J L (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  45. Dozat T, Manning C D (2016) Deep biaffine attention for neural dependency parsing. In: Proceedings of the 5th international conference on learning representations

  46. Liang Y, Meng F, Zhang J, Xu J, Chen Y, Zhou J (2019) A novel aspect-guided deep transition model for aspect based sentiment analysis. In: Proceedings of EMNLP-IJCNLP, pp 5568–5579

Download references

Acknowledgments

This work was supported by the Key Program of Science and Technology Research during the 13th Five-Year Plan Period, the Educational Department of Jilin Province of China (No. JJKH20200677KJ), the Youth Growth Science and Technology Plan Project of Jilin Provincial Department of Science and Technology (NO. 20210508039RQ). We thank the editor and reviewers for their valuable comments to improve the quality of this article.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Changchun University of Technology, Changchun, 130012, China

    Xiaotang Zhou, Tao Zhang, Chao Cheng & Shinan Song

  2. Artificial Intelligence Research Institute, Changchun University of Technology, Changchun, 130012, China

    Xiaotang Zhou & Chao Cheng

Authors
  1. Xiaotang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shinan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Chao Cheng or Shinan Song.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, X., Zhang, T., Cheng, C. et al. Dynamic multichannel fusion mechanism based on a graph attention network and BERT for aspect-based sentiment classification. Appl Intell 53, 6800–6813 (2023). https://doi.org/10.1007/s10489-022-03851-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03851-3

Keywords

Search

Navigation