Skip to main content
SpringerLink
Log in

Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

This paper proposes a state-of-the-art research for aspect-based sentiment analysis of Arabic Hotels’ reviews using two implementations of long short-term memory (LSTM) neural networks. The first one is (a) a character-level bidirectional LSTM along with conditional random field classifier (Bi-LSTM-CRF) for aspect opinion target expressions (OTEs) extraction, and the second one is (b) an aspect-based LSTM for aspect sentiment polarity classification in which the aspect-OTEs are considered as attention expressions to support the sentiment polarity identification. Proposed approaches are evaluated using a reference dataset of Arabic Hotels’ reviews. Results show that our approaches outperform baseline research on both tasks with an enhancement of 39% for the task of aspect-OTEs extraction and 6% for the aspect sentiment polarity classification task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://code.google.com/archive/p/word2vec/.

References

  1. Abdul-Mageed M, Diab M (2012) Toward building a large-scale Arabic sentiment lexicon. In: Proceedings of the 6th international global WordNet conference, pp 18–22

  2. Agerri R, Bermudez J, Rigau G (2014) Ixa pipeline: efficient and ready to use multilingual nlp tools. In: LREC, pp 3823–3828

  3. Al-Smadi M, Al-Ayyoub M, Al-Sarhan H, Jararweh Y (2015a) Using aspect-based sentiment analysis to evaluate arabic news affect on readers. In: 2015 IEEE/ACM 8th international conference on utility and cloud computing (UCC), IEEE, pp 436–441

  4. Al-Smadi M, Qawasmeh O, Talafha B, Quwaider M (2015b) Human annotated Arabic dataset of book reviews for aspect based sentiment analysis. In: future internet of things and cloud (FiCloud), 2015 3rd international conference, IEEE, pp 726–730

  5. Al-Smadi M, Al-Ayyoub M, Al-Sarhan H, Jararweh Y (2016a) An aspect-based sentiment analysis approach to evaluating Arabic news affect on readers. J Univers Comput Sci (JUCS) 22:630–649

    MathSciNet  Google Scholar 

  6. Al-Smadi M, Obaidat I, Al-Ayyoub M, Mohawesh R, Jararweh Y (2016b) Using enhanced lexicon-based approaches for the determination of aspect categories and their polarities in Arabic reviews. Int J Inf Technol Web Eng (IJITWE) 11(3):15–31

    Article  Google Scholar 

  7. Al-Smadi M, Qawasmeh O, Talafha B, Al-Ayyoub M, Jararweh Y, Benkhalifa E (2016c) An enhanced framework for aspect-based sentiment analysis of hotels reviews: Arabic reviews case study. In: International conference for internet technology and secured transactions (ICITST-2016), IEEE, pp 98–103

  8. Al-Twairesh N, Al-Khalifa H, Al-Salman A (2014) Subjectivity and sentiment analysis of Arabic: trends and challenges. In: 2014 IEEE/ACS 11th international conference on computer systems and applications (AICCSA), IEEE, pp 148–155

  9. Baccianella S, Esuli A, Sebastiani F (2010) Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. LREC 10:2200–2204

    Google Scholar 

  10. Badaro G, Baly R, Hajj H, Habash N, El-Hajj W (2014) A large scale arabic sentiment lexicon for Arabic opinion mining. ANLP 2014:165

    Google Scholar 

  11. Bastien F, Lamblin P, Pascanu R, Bergstra J, Goodfellow I, Bergeron A, Bouchard N, Warde-Farley D, Bengio Y (2012) Theano: new features and speed improvements. arXiv:1211.5590

  12. Bojanowski P, Grave E, Joulin A, Mikolov T (2016) Enriching word vectors with subword information. arXiv:1607.04606

  13. Brown PF, Desouza PV, Mercer RL, Pietra VJD, Lai JC (1992) Class-based n-gram models of natural language. Comput Linguist 18(4):467–479

    Google Scholar 

  14. Cambria E, Schuller B, Xia Y, Havasi C (2013) New avenues in opinion mining and sentiment analysis. IEEE Intell Syst 28(2):15–21

    Article  Google Scholar 

  15. Castellucci G, Filice S, Croce D, Basili R (2014) Unitor: aspect based sentiment analysis with structured learning. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), Citeseer, pp 761–767

  16. Chernyshevich M (2014) Ihs r&d belarus: cross-domain extraction of product features using conditional random fields. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 309–313

  17. Chevalier JA, Mayzlin D (2006) The effect of word of mouth on sales: online book reviews. J Mark Res 43(3):345–354

    Article  Google Scholar 

  18. Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder–decoder for statistical machine translation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Association for computational linguistics, Doha, pp 1724–1734. http://www.aclweb.org/anthology/D14-1179

  19. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555

  20. Chung J, Cho K, Bengio Y (2016) A character-level decoder without explicit segmentation for neural machine translation. In: Proceedings of the 54th annual meeting of the association for computational linguistics (Volume 1: Long Papers), Association for computational linguistics, Berlin, pp 1693–1703. http://www.aclweb.org/anthology/P16-1160

  21. Clark A (2003) Combining distributional and morphological information for part of speech induction. In: Proceedings of the 10th conference on European chapter of the Association for Computational Linguistics-Volume 1, Association for Computational Linguistics, pp 59–66

  22. Collins M (2002) Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms. In: Proceedings of the ACL-02 conference on empirical methods in natural language processing-Volume 10, Association for Computational Linguistics, pp 1–8

  23. Dahou A, Xiong S, Zhou J, Haddoud MH, Duan P (2016) Word embeddings and convolutional neural network for Arabic sentiment classification. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, The COLING 2016 Organizing Committee, Osaka, pp 2418–2427. http://aclweb.org/anthology/C16-1228

  24. Dean J, Corrado G, Monga R, Chen K, Devin M, Mao M, Senior A, Tucker P, Yang K, Le QV et al (2012) Large scale distributed deep networks. In: Advances in neural information processing systems, pp 1223–1231

  25. Fu X, Liu W, Xu Y, Cui L (2017) Combine hownet lexicon to train phrase recursive autoencoder for sentence-level sentiment analysis. Neurocomputing 241(Supplement C):18 – 27. https://doi.org/10.1016/j.neucom.2017.01.079. http://www.sciencedirect.com/science/article/pii/S0925231217302643

  26. Glorot X, Bordes A, Bengio Y (2011) Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 513–520

  27. Goller C, Kuchler A (1996) Learning task-dependent distributed representations by backpropagation through structure. In: Neural networks, 1996, IEEE international conference, IEEE, vol 1, pp 347–352

  28. Graves A (2013) Generating sequences with recurrent neural networks. arXiv:1308.0850

  29. Graves A, Schmidhuber J (2005) Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Netw 18(5):602–610

    Article  Google Scholar 

  30. Gridach M (2016) Character-aware neural networks for Arabic named entity recognition for social media. In: Proceedings of the 6th workshop on South and Southeast Asian natural language processing (WSSANLP2016), The COLING 2016 Organizing Committee, Osaka, pp 23–32. http://aclweb.org/anthology/W16-3703

  31. Habash NY (2010) Introduction to arabic natural language processing. Synth Lect Hum Lang Technol 3(1):1–187

    Article  Google Scholar 

  32. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  33. Hu M, Liu B (2004) Mining and summarizing customer reviews. In: Proceedings of the 10th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 168–177

  34. Irsoy O, Cardie C (2014) Opinion mining with deep recurrent neural networks. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Association for computational linguistics, Doha, pp 720–728. http://www.aclweb.org/anthology/D14-1080

  35. Kalchbrenner N, Danihelka I, Graves A (2015) Grid long short-term memory. arXiv:1507.01526

  36. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014a) Nrc-canada-2014: detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 437–442

  37. Kiritchenko S, Zhu X, Mohammad SM (2014b) Sentiment analysis of short informal texts. J Artif Intell Res 50:723–762

    Article  Google Scholar 

  38. Kumar A, Kohail S, Kumar A, Ekbal A, Biemann C (2016) Iit-tuda at semeval-2016 task 5: beyond sentiment lexicon: combining domain dependency and distributional semantics features for aspect based sentiment analysis. In: Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), Association for computational linguistics, San Diego, pp 1129–1135. http://www.aclweb.org/anthology/S16-1174

  39. Lafferty J, McCallum A, Pereira F et al (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the eighteenth international conference on machine learning, ICML, vol 1, pp 282–289

  40. Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C (2016) Neural architectures for named entity recognition. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, Association for computational linguistics, San Diego, pp 260–270. http://www.aclweb.org/anthology/N16-1030

  41. Li X, Xie H, Chen L, Wang J, Deng X (2014) News impact on stock price return via sentiment analysis. Knowl Based Syst 69:14–23

    Article  Google Scholar 

  42. Li X, Rao Y, Xie H, Lau RYK, Yin J, Wang FL (2017) Bootstrapping social emotion classification with semantically rich hybrid neural networks. IEEE Trans Affect Comput 8(4):428–442. https://doi.org/10.1109/TAFFC.2017.2716930

    Article  Google Scholar 

  43. Ling W, Dyer C, Black AW, Trancoso I, Fermandez R, Amir S, Marujo L, Luis T (2015) Finding function in form: Compositional character models for open vocabulary word representation. In: Proceedings of the 2015 conference on empirical methods in natural language processing, Association for computational linguistics, Lisbon, pp 1520–1530. http://aclweb.org/anthology/D15-1176

  44. Liou CY, Huang JC, Yang WC (2008) Modeling word perception using the elman network. Neurocomputing 71(16):3150–3157

    Article  Google Scholar 

  45. Liu B (2012) Sentiment analysis and opinion mining. Synth Lect Hum Lang Technol 5(1):1–167

    Article  Google Scholar 

  46. Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-Volume 1, Association for computational linguistics, pp 142–150

  47. Mahyoub FH, Siddiqui MA, Dahab MY (2014) Building an Arabic sentiment lexicon using semi-supervised learning. J King Saud Univ Comput Inf Sci 26(4):417–424

    Google Scholar 

  48. McCallum AK (2002) Mallet: a machine learning for language toolkit, Unpublished. http://mallet.cs.umass.edu

  49. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013a) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119

  50. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013b) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119

  51. Mohammad SM, Turney PD (2010) Emotions evoked by common words and phrases: using mechanical turk to create an emotion lexicon. In: Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and generation of emotion in text, Association for computational linguistics, pp 26–34

  52. Mohammad SM, Kiritchenko S, Zhu X (2013) Nrc-canada: building the state-of-the-art in sentiment analysis of tweets. arXiv:1308.6242

  53. Nguyen TH, Shirai K (2015) Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. In: Proceedings of the 2015 conference on empirical methods in natural language processing, Association for computational linguistics, Lisbon, pp 2509–2514. https://aclweb.org/anthology/D/D15/D15-1298

  54. Nielsen FÅ (2011) A new anew: evaluation of a word list for sentiment analysis in microblogs. arXiv:1103.2903

  55. Obaidat I, Mohawesh R, Al-Ayyoub M, Al-Smadi M, Jararweh Y (2015) Enhancing the determination of aspect categories and their polarities in Arabic reviews using lexicon-based approaches. In: Applied electrical engineering and computing technologies (AEECT), 2015 IEEE Jordan Conference, IEEE, pp 1–6

  56. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), Citeseer, pp 27–35

  57. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I (2015) Semeval-2015 task 12: aspect based sentiment analysis. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), Association for computational linguistics, Denver, pp 486–495

  58. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, Hoste V, Apidianaki M, Tannier X, Loukachevitch N, Kotelnikov E, Bel N, Jiménez-Zafra SM, Eryiğit G (2016) Semeval-2016 task 5: aspect based sentiment analysis. In: Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), Association for computational linguistics, pp 19–30. http://www.aclweb.org/anthology/S16-1002

  59. Popescu AM, Etzioni O (2007) Extracting product features and opinions from reviews. In: Natural language processing and text mining. Springer, London, pp 9–28

  60. Rao Y, Xie H, Li J, Jin F, Wang FL, Li Q (2016) Social emotion classification of short text via topic-level maximum entropy model. Inf Manag 53(8):978–986

    Article  Google Scholar 

  61. Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowl Based Syst 89:14–46

    Article  Google Scholar 

  62. Rosenthal S, Ritter A, Nakov P, Stoyanov V (2014) Semeval-2014 task 9: sentiment analysis in twitter. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 73–80

  63. Ruder S, Ghaffari P, Breslin JG (2016) Insight-1 at semeval-2016 task 5: deep learning for multilingual aspect-based sentiment analysis. arXiv:160902748

  64. Saias J (2015) Sentiue: target and aspect based sentiment analysis in semeval-2015 task 12. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Association for Computational Linguistics, pp 767–771

  65. Saif M, Mohammad MS, Kiritchenko S (2016) Sentiment lexicons for Arabic social media. In: Proceedings of 10th edition of the the language resources and evaluation conference (LREC), Portorož

  66. Salameh M, Mohammad S, Kiritchenko S (2015) Sentiment after translation: a case-study on Arabic social media posts. In: Proceedings of the 2015 conference of the North American chapter of the association for computational linguistics: human language technologies, Association for computational linguistics, Denver, pp 767–777. http://www.aclweb.org/anthology/N15-1078

  67. San Vicente I, Saralegi X, Agerri R, Sebastián DS (2015) Elixa: a modular and flexible absa platform. In: SemEval-2015, p 748

  68. dos Santos CN, Gatti M (2014) Deep convolutional neural networks for sentiment analysis of short texts. In: COLING, pp 69–78

  69. Scaffidi C, Bierhoff K, Chang E, Felker M, Ng H, Jin C (2007) Red opal: product-feature scoring from reviews. In: Proceedings of the 8th ACM conference on electronic commerce. ACM, pp 182–191

  70. Socher R, Perelygin A, Wu JY, Chuang J, Manning CD, Ng AY, Potts C et al (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the conference on empirical methods in natural language processing (EMNLP), Citeseer, vol 1631, p 1642

  71. Tamchyna A, Veselovská K (2016) Ufal at semeval-2016 task 5: recurrent neural networks for sentence classification. Proceedings of SemEval, pp 367–371

  72. Tang D, Wei F, Qin B, Liu T, Zhou M (2014) Coooolll: a deep learning system for twitter sentiment classification. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 208–212

  73. Tang D, Qin B, Feng X, Liu T (2016) Effective lstms for target-dependent sentiment classification. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, The COLING 2016 Organizing Committee, Osaka, pp 3298–3307. http://aclweb.org/anthology/C16-1311

  74. Todhunter J, Sovpel I, Pastanohau D (2013) System and method for automatic semantic labeling of natural language texts. US Patent 8,583,422

  75. Toh Z, Su J (2015) Nlangp: supervised machine learning system for aspect category classification and opinion target extraction, Association for Computational Linguistics, pp 496–501

  76. Toh Z, Wang W (2014) Dlirec: aspect term extraction and term polarity classification system. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 235–240

  77. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(Dec):3371–3408

    MathSciNet  MATH  Google Scholar 

  78. Vinodhini G, Chandrasekaran R (2016) A sampling based sentiment mining approach for e-commerce applications. Inf Process Manag 53(1):223–236

    Article  Google Scholar 

  79. Wagner J, Arora P, Cortes S, Barman U, Bogdanova D, Foster J, Tounsi L (2014) Dcu: aspect-based polarity classification for semeval task 4. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Association for Computational Linguistics and Dublin City University, pp 223–229

  80. Wang Y, Huang M, zhu x, Zhao L (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Association for computational linguistics, Austin, pp 606–615. https://aclweb.org/anthology/D16-1058

  81. Wiebe J, Wilson T, Cardie C (2005) Annotating expressions of opinions and emotions in language. Lang Resour Eval 39(2–3):165–210

    Article  Google Scholar 

  82. Wilson T, Hoffmann P, Somasundaran S, Kessler J, Wiebe J, Choi Y, Cardie C, Riloff E, Patwardhan S (2005) Opinionfinder: a system for subjectivity analysis. In: Proceedings of hlt/emnlp on interactive demonstrations, Association for Computational Linguistics, pp 34–35

  83. Yoo KH, Gretzel U (2008) What motivates consumers to write online travel reviews? Inf Technol Tour 10(4):283–295

    Article  Google Scholar 

  84. Zhang Z, Lan M (2015) Ecnu: extracting effective features from multiple sequential sentences for target-dependent sentiment analysis in reviews. In: SemEval-2015, p 736

Download references

Acknowledgements

This research is partially funded by Jordan University of Science and Technology, Research Grant number 20150164.

Author information

Authors and Affiliations

  1. Jordan University of Science and Technology, Irbid, Jordan

    Mohammad Al-Smadi, Bashar Talafha, Mahmoud Al-Ayyoub & Yaser Jararweh

Authors
  1. Mohammad Al-Smadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bashar Talafha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahmoud Al-Ayyoub
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yaser Jararweh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Al-Smadi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al-Smadi, M., Talafha, B., Al-Ayyoub, M. et al. Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews. Int. J. Mach. Learn. & Cyber. 10, 2163–2175 (2019). https://doi.org/10.1007/s13042-018-0799-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-018-0799-4

Keywords

Search

Navigation