Abstract
Aspect-based sentiment classification (ABSC) is a fine-grained analysis task that obtains different sentiment polarities contained in a single text from the views of different aspects. Its practicability draws so much attention from researchers that the number of related works grows explosively. However, existing works mainly aim to obtain polarities from short texts (shorter than 100 words), only a few works analyze documents (shorter than 500 words), but almost no work analyzes long documents (LD, longer than 500 words). This situation makes ABSC powerless when dealing with some texts like in-depth analysis articles. In this paper, we make ABSC step into the LD level by proposing the Hierarchical Aspect-Oriented Framework for Long Document (HAOFL). HAOFL solves two challenges that rarely appear in short texts and normal documents. The first is the too-long input sequence that can cause the model to forget previously learned information or ignore the tailed unlearned information. The second is the unstable sentiment information of the target aspect contained in LD, which increases the difficulty for a model to draw a proper result. HAOFL constructs the data transformation module, dependency processing module, and sentiment aggregation module to solve these two challenges. Numerical experiments prove HAOFL can solve the aforementioned challenges and achieve superior performance in an effective and resource-saving way. With HAOFL, the performances of popular ABSC models on LD are improved at most 8.69% of accuracy and 11.37% of F1-score. In terms of resource-consuming, up to 82.10% of training time and 71.03% of GPU memory are saved.
Similar content being viewed by others
Notes
References
Behdenna S, Barigou F, Belalem G (2018) Document level sentiment analysis: A survey. EAI Endorsed Trans Context-Aware Syst Appl 4(13)
Pang Bo, Lee L (2008) Opinion mining and sentiment analysis foundations and trends in information retrieval, vol 2
Liu B, Zhang L (2012) A survey of opinion mining and sentiment analysis. In: Mining text data. Springer, pp 415–463
Tang D, Qin B, Feng X, Liu T (2015) Effective lstms for target-dependent sentiment classification. arXiv:1512.01100
Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606–615
Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. arXiv:1709.00893
Chen P, Sun Z, Bing L, Yang W (2017) Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 452–461
Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 2514–2523
Wang Shuai, Mazumder S, Liu B, Zhou M, Chang Y (2018) Target-sensitive memory networks for aspect sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 957–967
He R, Lee WS, Ng HT, Dahlmeier D (2019) An interactive multi-task learning network for end-to-end aspect-based sentiment analysis. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 504–515
Hu M, Zhao S, Zhang L, Cai K, Su Z, Cheng R, Shen X (2019) Can: Constrained attention networks for multi-aspect sentiment analysis. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 4593–4602
Jiang L, Yu M, Zhou M, Liu X, Zhao T (2011) Target-dependent twitter sentiment classification. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pp 151–160
Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 437–442
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473
Ma Y, Peng H, Khan T, Cambria E, Hussain Amir (2018) Sentic lstm: a hybrid network for targeted aspect-based sentiment analysis. Cogn Comput 10(4):639–650
Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 946–956
Fan F, Feng Y, Zhao D (2018) Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 3433–3442
Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp 214–224
Zeng B, Yang H, Xu R, Zhou W, Han X (2019) Lcf: A local context focus mechanism for aspect-based sentiment classification. Applied Sciences 9(16):3389
Peng H, Ma Y, Li Y, Cambria E (2018) Learning multi-grained aspect target sequence for chinese sentiment analysis. Knowl-Based Syst 148:167–176
Ma Yukun, Peng Haiyun, Cambria Erik (2018) Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm.. In: Aaai, pp 5876–5883
Hussain A, Cambria E (2018) Semi-supervised learning for big social data analysis. Neurocomputing 275:1662–1673
Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) SemEval-2014 task 4: Aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Association for Computational Linguistics, Dublin, pp 27–35. https://www.aclweb.org/anthology/S14-2004
Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I (2015) Semeval-2015 task 12: Aspect based sentiment analysis. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pp 486–495
Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O et al (2016) Semeval-2016 task 5: Aspect based sentiment analysis. In: 10th International Workshop on Semantic Evaluation (SemEval 2016)
Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short papers), pp 49–54
Li J, Yang H, Zong C (2018) Document-level multi-aspect sentiment classification by jointly modeling users, aspects, and overall ratings. In: Proceedings of the 27th International Conference on Computational Linguistics, pp 925–936
Wang J, Sun C, Li S, Wang J, Si L, Zhang M, Liu X, Zhou G (2019) Human-like decision making: Document-level aspect sentiment classification via hierarchical reinforcement learning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 5585–5594
Zhang Q, Shi C (2019) An attentive memory network integrated with aspect dependency for document-level multi-aspect sentiment classification. In: Asian Conference on Machine Learning. PMLR, pp 425–440
Ji Y, Liu H, He B, Xiao X, Wu H, Yu Y (2020) Diversified multiple instance learning for document-level multi-aspect sentiment classification. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 7012–7023
Yang J, Yang R, Lu H, Wang C, Xie J (2019) Multi-entity aspect-based sentiment analysis with context, entity, aspect memory and dependency information. ACM Trans Asian Low-Resour Lang Inf Process (TALLIP) 18(4):1–22
Shi T, Rakesh V, Wang S, Reddy CK (2019) Document-level multi-aspect sentiment classification for online reviews of medical experts. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp 2723–2731
Joshi M, Chen D, Liu Y, Weld DS, Zettlemoyer L, Levy O (2020) Spanbert: Improving pre-training by representing and predicting spans. Trans Assoc Comput Linguist 8:64–77
Turney PD (2002) Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp 417–424
Zhang Q, Wang B, Wu L, Huang X (2007) Fdu at trec 2007: Opinion retrieval of blog track. In: TREC, pp 500–274
Tang D, Qin B, Liu T (2015) Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1422–1432
Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning. PMLR, pp 1188–1196
Chen H, Sun M, Tu C, Lin Y, Liu Z (2016) Neural sentiment classification with user and product attention. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 1650–1659
Zhou X, Wan X, Xiao J (2016) Attention-based lstm network for cross-lingual sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 247–256
Huang B, Ou Y, Carley KM (2018) Aspect level sentiment classification with attention-over-attention neural networks. In: International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. Springer, pp 197–206
Devlin J, Chang Mx-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805
Naseem U, Razzak I, Musial K, Imran M (2020) Transformer based deep intelligent contextual embedding for twitter sentiment analysis. Fut Gener Comput Syst 113:58–69
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Cohan A, Dernoncourt F, Kim DS, Bui T, Kim S, Chang W, Goharian N (2018) A discourse-aware attention model for abstractive summarization of long documents. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp 615–621
Wang Z, Ng P, Ma X, Nallapati R, Xiang B (2019) Multi-passage bert: A globally normalized bert model for open-domain question answering. arXiv:1908.08167
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wu, Z., Gao, J., Li, Q. et al. Make aspect-based sentiment classification go further: step into the long-document-level. Appl Intell 52, 8428–8447 (2022). https://doi.org/10.1007/s10489-021-02836-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02836-y