Abstract
Aspect Category Detection (ACD), which belongs to the research of fine-grained sentiment analysis, aims to identify the aspect categories mentioned in given sentences. However, the distribution of data from the real world is imbalanced or even long-tailed. This fact poses significant challenges for ACD because it is hard to fully extract the features of tail classes. Since a sentence usually contains one or more aspect categories, we model ACD as a multi-label text classification task. Under the long-tailed setting, this paper proposes a novel Aspect-Fusion model for Long-Tailed Aspect Category Detection (AFLoT-ACD). AFLoT-ACD first extracts the fine-grained aspect features from sentence vectors by the mechanism of Interactive Attention Network with characteristics of Long-Tailed distribution (IAN-LoT). A long-tailed distribution-based attention mechanism is also incorporated, which integrates contextual aspect-level semantic information. Additionally, an Advanced Distribution-Balanced loss (A-DB) is introduced to overcome the problems of label co-occurrence and the dominance of negative classes in training a long-tailed multi-label text classifier. We conduct experiments on three datasets and compare AFLoT-ACD with eight baselines. AFLoT-ACD outperforms the SOTA with over 7% improvements in Macro F1 score for tail classes and also achieves higher detection performance in general.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahmed, M., Chen, Q., Wang, Y., Li, Z.: Hint-embedding attention-based LSTM for aspect identification sentiment analysis. In: PRICAI 2019: Trends in Artificial Intelligence: 16th Pacific Rim International Conference on Artificial Intelligence, Cuvu, Yanuca Island, Fiji, August 26–30, 2019, Proceedings, Part II (2019)
Bengio, Y.: Courville, aaron, vincent, pascal: representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Buda, M., Maki, A., Mazurowski, M.A.: A systematic study of the class imbalance problem in convolutional neural networks. Neural Netw. 106, 249–259 (2018)
Dai, Z., Peng, C., Chen, H., Ding, Y.: A multi-task incremental learning framework with category name embedding for aspect-category sentiment analysis. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6955–6965 (2020)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota, June 2019. https://doi.org/10.18653/v1/N19-1423
Dilawar, N., et al.: Understanding citizen issues through reviews: a step towards data informed planning in smart cities. Appl. Sci. 8(9), 1589 (2018)
Ghadery, E., Movahedi, S., Faili, H., Shakery, A.: MNCN: a multilingual ngram-based convolutional network for aspect category detection in online reviews. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 6441–6448 (2019)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hu, M., et al.: Multi-label few-shot learning for aspect category detection. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 6330–6340 (2021)
Huang, B., Ou, Y., Carley, K.M.: Aspect level sentiment classification with attention-over-attention neural networks. In: International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation (2018)
Huang, Y., Giledereli, B., Köksal, A., Özgür, A., Ozkirimli, E.: Balancing methods for multi-label text classification with long-tailed class distribution. CoRR abs/2109.04712 (2021)
Huang, Z., Zhao, H., Peng, F., Chen, Q., Zhao, G.: Aspect category sentiment analysis with self-attention fusion networks. In: Nah, Y., Cui, B., Lee, S.-W., Yu, J.X., Moon, Y.-S., Whang, S.E. (eds.) DASFAA 2020. LNCS, vol. 12114, pp. 154–168. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59419-0_10
Jiang, Q., Chen, L., Xu, R., Ao, X., Yang, M.: A challenge dataset and effective models for aspect-based sentiment analysis. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (2019)
Kim.Y: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Li, Y., Yin, C., Zhong, S.: Sentence constituent-aware aspect-category sentiment analysis with graph attention networks. In: Zhu, X., Zhang, M., Hong, Yu., He, R. (eds.) NLPCC 2020. LNCS (LNAI), vol. 12430, pp. 815–827. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60450-9_64
Li, Y., Yin, C., Zhong, S.H., Pan, X.: Multi-instance multi-label learning networks for aspect-category sentiment analysis. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3550–3560 (2020)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 2999–3007 (2017)
Liu, Z., Miao, Z., Zhan, X., Wang, J., Gong, B., Yu, S.X.: Large-scale long-tailed recognition in an open world. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2537–2546 (2019)
Movahedi, S., Ghadery, E., Faili, H., Shakery, A.: Aspect category detection via topic-attention network. arXiv e-prints pp. arXiv-1901 (2019)
Paszke, A., et al.: Automatic differentiation in pytorch (2017)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., Manandhar, S.: Semeval-2014 task 4: aspect based sentiment analysis. In: Nakov, P., Zesch, T. (eds.) Proceedings of the 8th International Workshop on Semantic Evaluation, SemEval@COLING 2014, Dublin, Ireland, August 23–24, 2014. pp. 27–35. The Association for Computer Linguistics (2014)
Schouten, K., Weijde, O., Frasincar, F., Dekker, R.: Supervised and unsupervised aspect category detection for sentiment analysis with co-occurrence data. IEEE Trans. Cybern. 48(4), 1263–1275 (2018)
Siyu, W., Jiangtao, Q., Chuanyang, H.: A joint self-attention model for aspect category detection in e-commerce reviews. In: PACIS 2021 Proceedings (2021)
Wang, Y., Sun, A., Huang, M., Zhu, X.: Aspect-level sentiment analysis using as-capsules. In: The World Wide Web Conference, pp. 2033–2044 (2019)
Wu, T., Huang, Q., Liu, Z., Wang, Yu., Lin, D.: Distribution-balanced loss for multi-label classification in long-tailed datasets. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 162–178. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_10
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp. 1480–1489 (2016)
Yuan, M., Xu, J., Li, Z.: Long tail multi-label learning. In: 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), pp. 28–31. IEEE (2019)
Zhou, B., Cui, Q., Wei, X.S., Chen, Z.M.: BBN: bilateral-branch network with cumulative learning for long-tailed visual recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9719–9728 (2020)
Acknowledgements
This work was supported in part by the National Natural Science foundation of China under Grant 62073155, 62002137, and 62106088, in part by “Blue Project” in Jiangsu Universities, China, in part by Guangdong Provincial Key Laboratory under Grant 2020B121201001, in part by the China Postdoctoral Science Foundation under Grant 2022M711360.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nie, W., Lu, Hy., Fang, W. (2022). Pay Attention to the “Tails”: A Novel Aspect-Fusion Model for Long-Tailed Aspect Category Detection. In: Khanna, S., Cao, J., Bai, Q., Xu, G. (eds) PRICAI 2022: Trends in Artificial Intelligence. PRICAI 2022. Lecture Notes in Computer Science, vol 13630. Springer, Cham. https://doi.org/10.1007/978-3-031-20865-2_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-20865-2_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20864-5
Online ISBN: 978-3-031-20865-2
eBook Packages: Computer ScienceComputer Science (R0)