Skip to main content
SpringerLink
Log in

Ensemble application of bidirectional LSTM and GRU for aspect category detection with imbalanced data

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

E-commerce websites produce a large number of online reviews, posts, and comments about a product or service. These reviews are used to assist consumers in buying or recommending a product. However, consumers are expressing their views on a specific aspect category of a product. In particular, aspect category detection is one of the subtasks of aspect-based sentiment analysis, and it classifies a given text into a set of predefined aspects. Naturally, a class imbalance problem occurs in real-world applications. The class imbalance is studied over the last two decades using machine learning algorithms. However, there is very little empirical research in deep learning with the class imbalance problem. In this paper, we propose bidirectional LSTM and GRU networks to deal with imbalance aspect categories. The proposed method applies a data-level technique to reduce class imbalance. Specifically, we employ the stratified sampling technique to deal with imbalanced classes. Moreover, we create word vectors with the corpus-specific word embeddings and pre-trained word embeddings. This word representations fed into the proposed method and their merge modes such as addition, multiplication, average, and concatenation. The performance of this method is evaluated with a confusion matrix, precision, recall, F1-score with micro-average, macro-average, and weighted average. The experimental result analysis suggests that the proposed method outperforms with pre-trained word embeddings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Akhtar MS, Ekbal A, Cambria E (2020) How intense are you? Predicting intensities of emotions and sentiments using stacked ensemble. IEEE Comput Intell Mag 15(1):64–75

    Article  Google Scholar 

  2. Al-Smadi M, Talafha B, Al-Ayyoub M, Jararweh Y (2019) Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews. Int J Mach Learn Cybernet 10(8):2163–2175

    Article  Google Scholar 

  3. Basaldella M, Antolli E, Serra G, Tasso C (2018) Bidirectional lstm recurrent neural network for keyphrase extraction. In: Italian Research Conference on Digital Libraries, pp 180–187. Springer

  4. Basiri ME, Nemati S, Abdar M, Cambria E, Acharya UR (2021) ABCDM: an attention-based bidirectional cnn–rnn deep model for sentiment analysis. Future Gener Comput Syst 115:279–294

    Article  Google Scholar 

  5. Bronchal L (2017) Classifying with logistic regression (0.9473)—Kaggle. https://www.kaggle.com/lbronchal/classifying-with-logistic-regression-0-9473

  6. Cambria E, Wang H, White B (2014) Guest editorial: big social data analysis. Knowl Based Syst 69:1–2

    Article  Google Scholar 

  7. Chemchem A, Alin F, Krajecki M (2018) Deep learning and data mining classification through the intelligent agent reasoning. In: Proceedings of the 2018 6th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), pp 13–20. IEEE

  8. Cho K, van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: encoder–decoder approaches. In: Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pp 103–111. Association for Computational Linguistics, Doha, Qatar. https://doi.org/10.3115/v1/W14-4012.https://www.aclweb.org/anthology/W14-4012

  9. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014 Workshop on Deep Learning, December 2014

  10. Dua ED, Karra T (2017) UCI (University of California Irvine) Machine Learning Repository. Repository

  11. Ferri C, Hernández-Orallo J, Modroiu R (2009) An experimental comparison of performance measures for classification. Pattern Recogn Lett 30(1):27–38

    Article  Google Scholar 

  12. Gasparetti F (2017) Modeling user interests from web browsing activities. Data Min Knowl Disc 31(2):502–547

    Article  MathSciNet  Google Scholar 

  13. Ghadery E, Movahedi S, Sabet MJ, Faili H, Shakery A (2019) Licd: A language-independent approach for aspect category detection. In: European Conference on Information Retrieval, pp 575–589. Springer

  14. Graves A (2013) Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850

  15. Haixiang G, Yijing L, Shang J, Mingyun G, Yuanyue H, Bing G (2017) Learning from class-imbalanced data: Review of methods and applications. Expert Syst Appl 73:220–239

    Article  Google Scholar 

  16. Hinton G, Srivastava N, Swersky K (2012) Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited on 14(8)

  17. Hossin M, Sulaiman M (2015) A review on evaluation metrics for data classification evaluations. Int J Data Min Knowl Manag Process 5(2):1

    Article  Google Scholar 

  18. Hu M, Liu B (2004) Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pp 168–177

  19. Huang Z, Xu W, Yu K (2015) Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991

  20. Jayaraman AK, Abirami S (2020) Opinion-based co-occurrence network for identifying the most influential product features. J Eng Res 8(4)

  21. Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. IJCAI 16:1627–33

    Google Scholar 

  22. Jin Z, Yang Y, Liu Y (2019) Stock closing price prediction based on sentiment analysis and LSTM. Neural Comput Appl 32:1–17

    Google Scholar 

  23. Johnson JM, Khoshgoftaar TM (2019) Survey on deep learning with class imbalance. J Big Data 6(1):1–54

    Article  Google Scholar 

  24. Khan MU, Javed AR, Ihsan M, Tariq U (2020) A novel category detection of social media reviews in the restaurant industry. Multimedia Syst pp 1–14

  25. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) Nrc-Canada-2014: detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 437–442

  26. Li W, Liu P, Zhang Q, Liu W (2019) An improved approach for text sentiment classification based on a deep neural network via a sentiment attention mechanism. Future Internet 11(4):96

    Article  Google Scholar 

  27. Li Y, Wang S, Ma Y, Pan Q, Cambria E (2020) Popularity prediction on vacation rental websites. Neurocomputing 412:372–380

    Article  Google Scholar 

  28. Liao W, Zeng B, Yin X, Wei P (2020) An improved aspect-category sentiment analysis model for text sentiment analysis based on roberta. Appl Intell pp 1–12

  29. Liu N, Shen B (2020) Aspect-based sentiment analysis with gated alternate neural network. Knowl Based Syst 188

  30. Liu S, Ni’mah I, Menkovski V, Mocanu DC, Pechenizkiy M (2021) Efficient and effective training of sparse recurrent neural networks. Neural Comput Appl pp 1–12

  31. Ma Y, Peng H, Cambria E (2018) Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. AAAI, pp 5876–5883

  32. Ma Y, Peng H, Khan T, Cambria E, Hussain A (2018) Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis. Cognit Comput 10(4):639–650

    Article  Google Scholar 

  33. Majumder N, Bhardwaj R, Poria S, Gelbukh A, Hussain A (2020) Improving aspect-level sentiment analysis with aspect extraction. Neural Comput Appl pp 1–11

  34. Movahedi S, Ghadery E, Faili H, Shakery A (2019) Aspect category detection via topic-attention network. arXiv preprint arXiv:1901.01183

  35. Nagy P (2018) Multi class classification with LSTM. https://www.kaggle.com/ngyptr/multi-class-classification-with-lstm

  36. Park H, Song M, Shin KS (2020) Deep learning models and datasets for aspect term sentiment classification: implementing holistic recurrent attention on target-dependent memories. Knowl Based Syst 187:104825

    Article  Google Scholar 

  37. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  38. Pirzadeh H, Shanian S, Hamou-Lhadj A, Mehrabian A (2011) The concept of stratified sampling of execution traces. In: Proceedings of the 2011 IEEE 19th International Conference on Program Comprehension, pp 225–226. IEEE

  39. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, et al (2016) Semeval-2016 task 5: aspect based sentiment analysis. In: International workshop on semantic evaluation, pp 19–30

  40. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) SemEval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp 27–35. Association for Computational Linguistics, Dublin, Ireland. https://doi.org/10.3115/v1/S14-2004. https://www.aclweb.org/anthology/S14-2004

  41. Poria S, Cambria E, Gelbukh A (2016) Aspect extraction for opinion mining with a deep convolutional neural network. Knowl Based Syst 108:42–49

    Article  Google Scholar 

  42. Poria S, Cambria E, Ku LW, Gui C, Gelbukh A (2014) A rule-based approach to aspect extraction from product reviews. In: Proceedings of the second workshop on natural language processing for social media (SocialNLP), pp 28–37

  43. Pushp PK, Srivastava MM (2017) Train once, test anywhere: zero-shot learning for text classification. arXiv preprint arXiv:1712.05972

  44. Qiu G, Liu B, Bu J, Chen C (2011) Opinion word expansion and target extraction through double propagation. Comput Linguist 37(1):9–27

    Article  Google Scholar 

  45. Raj D, Sahu S, Anand A (2017) Learning local and global contexts using a convolutional recurrent network model for relation classification in biomedical text. In: Proceedings of the 21st conference on computational natural language learning (CoNLL 2017), pp 311–321

  46. Sahu S, Anand A (2016) Recurrent neural network models for disease name recognition using domain invariant features. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2216–2225. Association for Computational Linguistics, Berlin, Germany. doi: https://doi.org/10.18653/v1/P16-1209. https://www.aclweb.org/anthology/P16-1209

  47. Schmidhuber J, Hochreiter S (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  48. Schouten K, Van Der Weijde O, Frasincar F, Dekker R (2017) Supervised and unsupervised aspect category detection for sentiment analysis with co-occurrence data. IEEE Trans Cybern 48(4):1263–1275

    Article  Google Scholar 

  49. Toh Z, Su J (2016) Nlangp at semeval-2016 task 5: improving aspect based sentiment analysis using neural network features. In: Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), pp 282–288

  50. Zhou X, Wan X, Xiao J (2015) Representation learning for aspect category detection in online reviews. In: Proceedings of the AAAI conference on artificial intelligence, vol. 29

Download references

Acknowledgements

We thank the University Grants Commission (UGC), Government of India for supporting this work under the UGC National Fellowship. Also, we thank editors and reviewers for their valuable comments and guidance to improve the earlier version of this paper.

Author information

Authors and Affiliations

  1. Department of Information Science and Technology, Anna University, Chennai, 600025, India

    J. Ashok Kumar & S. Abirami

Authors
  1. J. Ashok Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Abirami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Ashok Kumar.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, J.A., Abirami, S. Ensemble application of bidirectional LSTM and GRU for aspect category detection with imbalanced data. Neural Comput & Applic 33, 14603–14621 (2021). https://doi.org/10.1007/s00521-021-06100-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06100-9

Keywords

Search

Navigation