Skip to main content
SpringerLink
Log in

Cooperative gating network based on a single BERT encoder for aspect term sentiment analysis

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In recent years, BERT encoder methods have been widely used in aspect term sentiment analysis (ATSA) tasks. Many ways of putting text and aspect term into a BERT sentence encoder separately aim to create vectors by obtaining context and aspect words. However, the semantic relevance of these initially extracted context-hiding vectors and aspect word-hiding vectors is poor. Moreover, they are easily affected by irrelevant words. Therefore, the CGBN model is proposed in this paper, which uses only the sentence sequence as the input to the BERT encoder. Moreover, the context-hiding vectors and aspect word-hiding vectors containing rich semantic association information were able to be extracted simultaneously for the first time. In addition, this paper proposes a new interactive gating mechanism called a co-gate. Compared with the general interactive feature extraction mechanism, it can not only effectively reduce the interference of noisy words but also fuse the information of context and aspect term better and capture emotional semantic features. To enhance the ability of BERT to be fine-tuned with domain data, the pretraining file of BERT Post Training (BERT-PT) is used in this paper to fine-tune the CGBN model. A method of domain adaptation is also applied with combined training sets, thus enhancing the training effect of the target domain data. Experiments and analysis prove the validity of the model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Rana TA, Cheah Y-N (2016) Aspect extraction in sentiment analysis: comparative analysis and survey. Artif Intell Rev 46:459–483. https://doi.org/10.1007/s10462-016-9472-z

    Article  Google Scholar 

  2. Appel O, Chiclana F, Carter J, Fujita H (2017) A consensus approach to the sentiment analysis problem driven by support-based IOWA majority. Int J Intell Syst 32:947–965. https://doi.org/10.1002/int.21878

    Article  Google Scholar 

  3. Appel O, Chiclana F, Carter J, Fujita H (2017) Cross-ratio uninorms as an effective aggregation mechanism in sentiment analysis. Knowl Based Syst 124:16–22. https://doi.org/10.1016/j.knosys.2017.02.028

    Article  Google Scholar 

  4. Dosoula N, Griep R, Den Ridder R et al (2016) Sentiment analysis of multiple implicit features per sentence in consumer review data. Front Artif Intell Appl:241–254. https://doi.org/10.3233/978-1-61499-714-6-241

  5. Do HH, Prasad P, Maag A, Alsadoon A (2019) Deep learning for aspect-based sentiment analysis: a comparative review. Expert Syst Appl 118:272–299. https://doi.org/10.1016/j.eswa.2018.10.003

    Article  Google Scholar 

  6. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th annual meeting of the association for computational linguistics, vol 2018, pp 2514–2523

  7. Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  8. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv:1301.3781

  9. J Devlin, M-W Chang, K Lee, K Toutanova (2019) BERT: Pre-training of deep bidirectional transformers for language Understanding, Proc. NAACL-HLT, pp 4171–4186 2019

  10. Vaswani A, Brain G, Shazeer N et al (2017) Attention is all you need. Adv Neural Inf Process Syst 30:5998–6008

    Google Scholar 

  11. Su J, Yu S, Luo D (2020) Enhancing aspect-based sentiment analysis with capsule network. IEEE Access 8:100551–100561. https://doi.org/10.1109/ACCESS.2020.2997675

    Article  Google Scholar 

  12. Yang C, Zhang H, Jiang B, Li K (2019) Aspect-based sentiment analysis with alternating coattention networks. Inf Process Manag 56:463–478. https://doi.org/10.1016/j.ipm.2018.12.004

    Article  Google Scholar 

  13. He R, Lee WS, Ng HT, Dahlmeier D (2018) Exploiting document knowledge for aspect-level sentiment classification. In: Proceedings of the 56th annual meeting of the Association for Computational Linguistics (volume 2: short papers). Association for Computational Linguistics, Stroudsburg, PA, USA, pp 579–585

  14. Hu X, Bing L, Lei S, Philip YS (2019) BERT post-training for review reading comprehension and aspect-based sentiment analysis. Proc NAACL:2324–2335

  15. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, vol 2: short papers, pp 49–54

  16. Park H, Song M, Shin K-S (2020) Deep learning models and datasets for aspect term sentiment classification: implementing holistic recurrent attention on target-dependent memories. Knowl Based Syst 187:104825. https://doi.org/10.1016/j.knosys.2019.06.033

    Article  Google Scholar 

  17. Shuang K, Yang Q, Loo J, Li R, Gu M (2020) Feature distillation network for aspect-based sentiment analysis. Inf Fusion 61:13–23. https://doi.org/10.1016/j.inffus.2020.03.003

    Article  Google Scholar 

  18. Lin Y, Wang C, Song H, Li Y (2021) Multi-head self-attention transformation networks for aspect-based sentiment analysis. IEEE Access 9:8762–8770. https://doi.org/10.1109/ACCESS.2021.3049294

    Article  Google Scholar 

  19. Wang K, Shen W, Yang Y, et al (2020) Relational graph attention network for aspect-based sentiment analysis. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 3229–3238

  20. Shuang K, Gu M, Li R, Loo J, Su S (2021) Interactive POS-aware network for aspect-level sentiment classification. Neurocomputing 420:181–196. https://doi.org/10.1016/j.neucom.2020.08.013

    Article  Google Scholar 

  21. Lv Y, Wei F, Cao L et al (2021) Aspect-level sentiment analysis using context and aspect memory network. Neurocomputing 428:195–205. https://doi.org/10.1016/j.neucom.2020.11.049

    Article  Google Scholar 

  22. Wu C, Xiong Q, Yang Z, Gao M, Li Q, Yu Y, Wang K, Zhu Q (2021) Residual attention and other aspects module for aspect-based sentiment analysis. Neurocomputing. 435:42–52. https://doi.org/10.1016/j.neucom.2021.01.019

    Article  Google Scholar 

  23. Liu MZ, Zhou FY, Chen K, Zhao Y (2021) Co-attention networks based on aspect and context for aspect-level sentiment analysis. Knowl Based Syst 217:106810. https://doi.org/10.1016/j.knosys.2021.106810

    Article  Google Scholar 

  24. Zhang Q, Lu R, Wang Q, Zhu Z, Liu P (2019) Interactive multi-head attention networks for aspect-level sentiment classification. IEEE Access 7:160017–160028. https://doi.org/10.1109/ACCESS.2019.2951283

    Article  Google Scholar 

  25. Fan F, Feng Y, Zhao D (2018) Multi-grained Attention Network for Aspect-Level Sentiment Classification. In: Proceedings of the 2018 Conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 3433–3442

  26. Song Y, Wang J, Jiang T et al (2019) Targeted sentiment classification with attentional encoder network. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), pp 93–103

  27. Li X, Fu X, Xu G, Yang Y, Wang J, Jin L, Liu Q, Xiang T (2020) Enhancing BERT representation with context-aware embedding for aspect-based sentiment analysis. IEEE Access 8:46868–46876. https://doi.org/10.1109/ACCESS.2020.2978511

    Article  Google Scholar 

  28. Kumar A, Narapareddy VT, Aditya Srikanth V, Neti LBM, Malapati A (2020) Aspect-based sentiment classification using interactive gated convolutional network. IEEE Access 8:22445–22453. https://doi.org/10.1109/ACCESS.2020.2970030

    Article  Google Scholar 

  29. Sun K, Zhang R, Mensah S et al (2019) Aspect-level sentiment analysis via convolution over dependency tree. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 5678–5687

  30. Zhang C, Li Q, Song D (2019) Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 4567–4577

  31. Chen F, Huang Y (2019) Knowledge-enhanced neural networks for sentiment analysis of Chinese reviews. Neurocomputing. 368:51–58. https://doi.org/10.1016/j.neucom.2019.08.054

    Article  Google Scholar 

  32. Peters M, Neumann M, Iyyer M et al (2018) Deep contextualized word representations. In: Proceedings of the 2018 conference of the north American chapter of the Association for Computational Linguistics: human language technologies, volume 1 (long papers), pp 2227–2237

  33. Peters ME, Neumann M, Iyyer M et al (2018) Improving language understanding by generative pre-training. OpenAI

    Google Scholar 

  34. Yang Z, Dai Z, Yang Y et al (2019) XLNet: generalized autoregressive pretraining for language understanding. In: Advances in Neural Information Processing Systems, pp 5754–5764

  35. Rietzler A, Stabinger S, Opitz P, Engl S (2019) Adapt or get left behind: domain adaptation through BERT language model Finetuning for aspect-target sentiment classification. Proc 12th Conf Lang Resour Eval (LREC 2020) 4933–4941

  36. Rana TA, Cheah Y-N (2017) A two-fold rule-based model for aspect extraction. Expert Syst Appl 89:273–285. https://doi.org/10.1016/j.eswa.2017.07.047

    Article  Google Scholar 

  37. Gao Z, Feng A, Song X, Wu X (2019) Target-dependent sentiment classification with BERT. IEEE Access 7:154290–154299. https://doi.org/10.1109/ACCESS.2019.2946594

    Article  Google Scholar 

  38. Zeng B, Yang H, Xu R, Zhou W, Han X (2019) LCF: a local context focus mechanism for aspect-based sentiment classification. Appl Sci 9:3389. https://doi.org/10.3390/app9163389

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Post-graduate’s Innovation Fund Project of Hebei Province (No. CXZZSS2021043) and in part by Natural Science Foundation of Hebei Province (No. F2021202038).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Hebei University of Technology, Tianjin, China

    Yuqing Peng, Tengfei Xiao & Hongtao Yuan

Authors
  1. Yuqing Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tengfei Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongtao Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuqing Peng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, ., Xiao, T. & Yuan, H. Cooperative gating network based on a single BERT encoder for aspect term sentiment analysis. Appl Intell 52, 5867–5879 (2022). https://doi.org/10.1007/s10489-021-02724-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02724-5

Keywords

Search

Navigation