Advertisement

Neural Processing Letters

, Volume 50, Issue 3, pp 2745–2761 | Cite as

A Robust Sentiment Analysis Method Based on Sequential Combination of Convolutional and Recursive Neural Networks

  • Hossein Sadr
  • Mir Mohsen PedramEmail author
  • Mohammad Teshnehlab
Article
  • 78 Downloads

Abstract

With explosive development of the World Wide Web, an enormous amount of text information containing users’ feeling, emotions and opinions has been generated and is increasingly employed by individuals and companies for making decisions. Whereas unstructured form of data must be analyzed to extract and summarize the opinions in them, sentiment analysis has changed to a significant research area in the field of Natural Language Processing. In this regard, deep learning methods have attracted a lot of attentions in recent years and various deep learning models have been proven as effective network architectures for the task of sentiment analysis. However, each of them has its potentials and weak points. To eliminate their drawbacks and make optimal use of their benefits, convolutional and recursive neural network are merged into a new robust model in this paper. The proposed model employs recursive neural network due to its tree structure as a substitute of pooling layer in the convolutional network with the aim of capturing long-term dependencies and reducing the loss of local information. The proposed model is validated on Stanford Sentiment Treebank by conducting a series of experiments and empirical results revealed that our model outperforms basic convolutional and recursive neural networks while requires fewer parameters.

Keywords

Deep learning Sentiment analysis Convolutional neural network Recursive neural network Combinational model 

Notes

References

  1. 1.
    Cambria E, Poria S, Gelbukh A, Thelwall M (2017) Sentiment analysis is a big suitcase. IEEE Intell Syst 32(6):74–80CrossRefGoogle Scholar
  2. 2.
    Vyas V, Uma V (2019) Approaches to sentiment analysis on product reviews. In: Sentiment analysis and knowledge discovery in contemporary business. IGI Global, pp 15–30Google Scholar
  3. 3.
    Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowl-Based Syst 89:14–46CrossRefGoogle Scholar
  4. 4.
    Souma W, Vodenska I, Aoyama H (2019) Enhanced news sentiment analysis using deep learning methods. J Comput Soc Sci 2:1–14CrossRefGoogle Scholar
  5. 5.
    Ain QT, Ali M, Riaz A, Noureen A, Kamran M, Hayat B, Rehman A (2017) Sentiment analysis using deep learning techniques: a review. Int J Adv Comput Sci Appl 8(6):424Google Scholar
  6. 6.
    Sadr H, Nazari Solimandarabi M (2019) Presentation of an efficient automatic short answer grading model based on combination of pseudo relevance feedback and semantic relatedness measures. J Adv Comput Res 10(2):1–10Google Scholar
  7. 7.
    Jadidinejad A, Sadr H (2015) Improving weak queries using local cluster analysis as a preliminary framework. Indian J Sci Technol 8:15CrossRefGoogle Scholar
  8. 8.
    Abirami A, Gayathri V (2017) A survey on sentiment analysis methods and approach. In: 8th International conference on advanced computing (ICoAC), IEEE, pp 72–76Google Scholar
  9. 9.
    Medhat W, Hassan A, Korashy H (2014) Sentiment analysis algorithms and applications: a survey. Ain Shams Eng J 5(4):1093–1113CrossRefGoogle Scholar
  10. 10.
    Soleimandarabi MN, Mirroshandel SA (2015) A novel approach for computing semantic relatedness of geographic terms. Indian J Sci Technol 8:27CrossRefGoogle Scholar
  11. 11.
    Soleimandarabi MN, Mirroshandel SA, Sadr H (2015) A Survey of semantic relatedness measures. Int J Comput Sci Network Solut 3(2):243–247Google Scholar
  12. 12.
    Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75CrossRefGoogle Scholar
  13. 13.
    Mohammad SM (2017) Challenges in sentiment analysis. In: A practical guide to sentiment analysis. Springer, pp 61–83Google Scholar
  14. 14.
    Ouyang X, Zhou P, Li CH, Liu L (2015) Sentiment analysis using convolutional neural network. In: 2015 IEEE International conference on computer and information technology; ubiquitous computing and communications; dependable, autonomic and secure computing; pervasive intelligence and computing. IEEE, pp 2359–2364Google Scholar
  15. 15.
    Hassan A, Mahmood A (2017) Deep learning approach for sentiment analysis of short texts. In: 3rd International conference on control, automation and robotics (ICCAR), IEEE, pp 705–710Google Scholar
  16. 16.
    Van VD, Thai T, Nghiem M-Q (2017) Combining convolution and recursive neural networks for sentiment analysis. In: Proceedings of the 8th international symposium on information and communication technology, 2017. ACM, pp 151–158Google Scholar
  17. 17.
    Chen D, Zhang Y (2018) Robust zeroing neural-dynamics and its time-varying disturbances suppression model applied to mobile robot manipulators. IEEE Trans Neural Netw Learn Syst 29(9):4385–4397CrossRefGoogle Scholar
  18. 18.
    Chen K, Yi C (2016) Robustness analysis of a hybrid of recursive neural dynamics for online matrix inversion. Appl Math Comput 273:969–975MathSciNetzbMATHGoogle Scholar
  19. 19.
    Jin L, Li S, Hu B (2018) RNN models for dynamic matrix inversion: a control-theoretical perspective. IEEE Trans Ind Inf 14(1):189–199CrossRefGoogle Scholar
  20. 20.
    You Q, Cao L, Jin H, Luo J (2016) Robust visual-textual sentiment analysis: When attention meets tree-structured recursive neural networks. In: Proceedings of the 24th ACM international conference on multimedia, ACM, pp 1008–1017Google Scholar
  21. 21.
    Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning, pp 1188–1196Google Scholar
  22. 22.
    Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-volume 1, Association for Computational Linguistics, pp 142–150Google Scholar
  23. 23.
    Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119Google Scholar
  24. 24.
    Islam J, Zhang Y (2016) Visual sentiment analysis for social images using transfer learning approach. In: 2016 IEEE international conferences on big data and cloud computing (BDCloud), social computing and networking (SocialCom), sustainable computing and communications (SustainCom)(BDCloud-SocialCom-SustainCom), IEEE, pp 124–130Google Scholar
  25. 25.
    Kim Y (2014) Convolutional neural networks for sentence classification. arXiv:14085882
  26. 26.
    Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification. In: Advances in neural information processing systems, pp 649-657Google Scholar
  27. 27.
    Socher R, Lin CC, Manning C, Ng AY (2011) Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 129–136Google Scholar
  28. 28.
    Socher R, Huval B, Manning CD, Ng AY (2012) Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, association for computational linguistics, pp 1201–1211Google Scholar
  29. 29.
    Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1631–1642Google Scholar
  30. 30.
    Wang X, Jiang W, Luo Z (2016) Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, pp 2428–2437Google Scholar
  31. 31.
    Huang Q, Chen R, Zheng X, Dong Z (2017) Deep sentiment representation based on CNN and LSTM. In: 2017 International conference on green informatics (ICGI), IEEE, pp 30–33Google Scholar
  32. 32.
    Timmaraju A, Khanna V (2015) Sentiment analysis on movie reviews using recursive and recurrent neural network architectures. Semantic ScholarGoogle Scholar
  33. 33.
    Jin L, Li S, Hu B, Liu M, Yu J (2019) A noise-suppressing neural algorithm for solving the time-varying system of linear equations: a control-based approach. IEEE Trans Ind Inf 15(1):236–246CrossRefGoogle Scholar
  34. 34.
    Chen D, Li S, Wu Q (2019) Rejecting chaotic disturbances using a super-exponential-zeroing neurodynamic approach for synchronization of chaotic sensor systems. Sensors 19(1):74CrossRefGoogle Scholar
  35. 35.
    Chen D, Zhang Y, Li S (2018) Tracking control of robot manipulators with unknown models: a Jacobian-matrix-adaption method. IEEE Trans Ind Inf 14(7):3044–3053CrossRefGoogle Scholar
  36. 36.
    Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. arXiv:14042188
  37. 37.
    Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537zbMATHGoogle Scholar
  38. 38.
    Uličný M, Lundström J, Byttner S (2016) Robustness of deep convolutional neural networks for image recognition. In: International symposium on intelligent computing systems. Springer, pp 16–30Google Scholar
  39. 39.
    Park S, Kwak N (2016) Analysis on the dropout effect in convolutional neural networks. In: Asian conference on computer vision. Springer, pp 189–204Google Scholar
  40. 40.
    Du C, Huang L (2017) Sentiment classification via recurrent convolutional neural networks. DEStech Trans Comput Scie Eng (cii)Google Scholar
  41. 41.
    Yin W, Schütze H (2016) Multichannel variable-size convolution for sentence classification. arXiv:160304513
  42. 42.
    Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. arXiv:150300075
  43. 43.
    Kokkinos F, Potamianos A (2017) Structural attention neural networks for improved sentiment analysis. arXiv:170101811
  44. 44.
    Wang Y, Huang M, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606-615Google Scholar
  45. 45.
    Sharif M, Abadi HK (2018) Recursive nested neural network for sentiment analysis. Stanford University ReportGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Hossein Sadr
    • 1
  • Mir Mohsen Pedram
    • 2
    Email author
  • Mohammad Teshnehlab
    • 3
  1. 1.Department of Computer Engineering, Rasht BranchIslamic Azad UniversityRashtIran
  2. 2.Department of Electrical and Computer Engineering, Faculty of EngineeringKharazmi UniversityTehranIran
  3. 3.Industrial Control Center of Excellence, Faculty of Electrical and Computer EngineeringK. N. Toosi UniversityTehranIran

Personalised recommendations