Skip to main content
SpringerLink
Log in

A Robust Sentiment Analysis Method Based on Sequential Combination of Convolutional and Recursive Neural Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

With explosive development of the World Wide Web, an enormous amount of text information containing users’ feeling, emotions and opinions has been generated and is increasingly employed by individuals and companies for making decisions. Whereas unstructured form of data must be analyzed to extract and summarize the opinions in them, sentiment analysis has changed to a significant research area in the field of Natural Language Processing. In this regard, deep learning methods have attracted a lot of attentions in recent years and various deep learning models have been proven as effective network architectures for the task of sentiment analysis. However, each of them has its potentials and weak points. To eliminate their drawbacks and make optimal use of their benefits, convolutional and recursive neural network are merged into a new robust model in this paper. The proposed model employs recursive neural network due to its tree structure as a substitute of pooling layer in the convolutional network with the aim of capturing long-term dependencies and reducing the loss of local information. The proposed model is validated on Stanford Sentiment Treebank by conducting a series of experiments and empirical results revealed that our model outperforms basic convolutional and recursive neural networks while requires fewer parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cambria E, Poria S, Gelbukh A, Thelwall M (2017) Sentiment analysis is a big suitcase. IEEE Intell Syst 32(6):74–80

    Article  Google Scholar 

  2. Vyas V, Uma V (2019) Approaches to sentiment analysis on product reviews. In: Sentiment analysis and knowledge discovery in contemporary business. IGI Global, pp 15–30

  3. Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowl-Based Syst 89:14–46

    Article  Google Scholar 

  4. Souma W, Vodenska I, Aoyama H (2019) Enhanced news sentiment analysis using deep learning methods. J Comput Soc Sci 2:1–14

    Article  Google Scholar 

  5. Ain QT, Ali M, Riaz A, Noureen A, Kamran M, Hayat B, Rehman A (2017) Sentiment analysis using deep learning techniques: a review. Int J Adv Comput Sci Appl 8(6):424

    Google Scholar 

  6. Sadr H, Nazari Solimandarabi M (2019) Presentation of an efficient automatic short answer grading model based on combination of pseudo relevance feedback and semantic relatedness measures. J Adv Comput Res 10(2):1–10

    Google Scholar 

  7. Jadidinejad A, Sadr H (2015) Improving weak queries using local cluster analysis as a preliminary framework. Indian J Sci Technol 8:15

    Article  Google Scholar 

  8. Abirami A, Gayathri V (2017) A survey on sentiment analysis methods and approach. In: 8th International conference on advanced computing (ICoAC), IEEE, pp 72–76

  9. Medhat W, Hassan A, Korashy H (2014) Sentiment analysis algorithms and applications: a survey. Ain Shams Eng J 5(4):1093–1113

    Article  Google Scholar 

  10. Soleimandarabi MN, Mirroshandel SA (2015) A novel approach for computing semantic relatedness of geographic terms. Indian J Sci Technol 8:27

    Article  Google Scholar 

  11. Soleimandarabi MN, Mirroshandel SA, Sadr H (2015) A Survey of semantic relatedness measures. Int J Comput Sci Network Solut 3(2):243–247

    Google Scholar 

  12. Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75

    Article  Google Scholar 

  13. Mohammad SM (2017) Challenges in sentiment analysis. In: A practical guide to sentiment analysis. Springer, pp 61–83

  14. Ouyang X, Zhou P, Li CH, Liu L (2015) Sentiment analysis using convolutional neural network. In: 2015 IEEE International conference on computer and information technology; ubiquitous computing and communications; dependable, autonomic and secure computing; pervasive intelligence and computing. IEEE, pp 2359–2364

  15. Hassan A, Mahmood A (2017) Deep learning approach for sentiment analysis of short texts. In: 3rd International conference on control, automation and robotics (ICCAR), IEEE, pp 705–710

  16. Van VD, Thai T, Nghiem M-Q (2017) Combining convolution and recursive neural networks for sentiment analysis. In: Proceedings of the 8th international symposium on information and communication technology, 2017. ACM, pp 151–158

  17. Chen D, Zhang Y (2018) Robust zeroing neural-dynamics and its time-varying disturbances suppression model applied to mobile robot manipulators. IEEE Trans Neural Netw Learn Syst 29(9):4385–4397

    Article  Google Scholar 

  18. Chen K, Yi C (2016) Robustness analysis of a hybrid of recursive neural dynamics for online matrix inversion. Appl Math Comput 273:969–975

    MathSciNet  MATH  Google Scholar 

  19. Jin L, Li S, Hu B (2018) RNN models for dynamic matrix inversion: a control-theoretical perspective. IEEE Trans Ind Inf 14(1):189–199

    Article  Google Scholar 

  20. You Q, Cao L, Jin H, Luo J (2016) Robust visual-textual sentiment analysis: When attention meets tree-structured recursive neural networks. In: Proceedings of the 24th ACM international conference on multimedia, ACM, pp 1008–1017

  21. Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning, pp 1188–1196

  22. Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies-volume 1, Association for Computational Linguistics, pp 142–150

  23. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119

  24. Islam J, Zhang Y (2016) Visual sentiment analysis for social images using transfer learning approach. In: 2016 IEEE international conferences on big data and cloud computing (BDCloud), social computing and networking (SocialCom), sustainable computing and communications (SustainCom)(BDCloud-SocialCom-SustainCom), IEEE, pp 124–130

  25. Kim Y (2014) Convolutional neural networks for sentence classification. arXiv:14085882

  26. Zhang X, Zhao J, LeCun Y (2015) Character-level convolutional networks for text classification. In: Advances in neural information processing systems, pp 649-657

  27. Socher R, Lin CC, Manning C, Ng AY (2011) Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 129–136

  28. Socher R, Huval B, Manning CD, Ng AY (2012) Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, association for computational linguistics, pp 1201–1211

  29. Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1631–1642

  30. Wang X, Jiang W, Luo Z (2016) Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, pp 2428–2437

  31. Huang Q, Chen R, Zheng X, Dong Z (2017) Deep sentiment representation based on CNN and LSTM. In: 2017 International conference on green informatics (ICGI), IEEE, pp 30–33

  32. Timmaraju A, Khanna V (2015) Sentiment analysis on movie reviews using recursive and recurrent neural network architectures. Semantic Scholar

  33. Jin L, Li S, Hu B, Liu M, Yu J (2019) A noise-suppressing neural algorithm for solving the time-varying system of linear equations: a control-based approach. IEEE Trans Ind Inf 15(1):236–246

    Article  Google Scholar 

  34. Chen D, Li S, Wu Q (2019) Rejecting chaotic disturbances using a super-exponential-zeroing neurodynamic approach for synchronization of chaotic sensor systems. Sensors 19(1):74

    Article  Google Scholar 

  35. Chen D, Zhang Y, Li S (2018) Tracking control of robot manipulators with unknown models: a Jacobian-matrix-adaption method. IEEE Trans Ind Inf 14(7):3044–3053

    Article  Google Scholar 

  36. Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. arXiv:14042188

  37. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537

    MATH  Google Scholar 

  38. Uličný M, Lundström J, Byttner S (2016) Robustness of deep convolutional neural networks for image recognition. In: International symposium on intelligent computing systems. Springer, pp 16–30

  39. Park S, Kwak N (2016) Analysis on the dropout effect in convolutional neural networks. In: Asian conference on computer vision. Springer, pp 189–204

  40. Du C, Huang L (2017) Sentiment classification via recurrent convolutional neural networks. DEStech Trans Comput Scie Eng (cii)

  41. Yin W, Schütze H (2016) Multichannel variable-size convolution for sentence classification. arXiv:160304513

  42. Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. arXiv:150300075

  43. Kokkinos F, Potamianos A (2017) Structural attention neural networks for improved sentiment analysis. arXiv:170101811

  44. Wang Y, Huang M, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606-615

  45. Sharif M, Abadi HK (2018) Recursive nested neural network for sentiment analysis. Stanford University Report

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Rasht Branch, Islamic Azad University, Rasht, Iran

    Hossein Sadr

  2. Department of Electrical and Computer Engineering, Faculty of Engineering, Kharazmi University, Tehran, Iran

    Mir Mohsen Pedram

  3. Industrial Control Center of Excellence, Faculty of Electrical and Computer Engineering, K. N. Toosi University, Tehran, Iran

    Mohammad Teshnehlab

Authors
  1. Hossein Sadr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mir Mohsen Pedram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Teshnehlab
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mir Mohsen Pedram.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sadr, H., Pedram, M.M. & Teshnehlab, M. A Robust Sentiment Analysis Method Based on Sequential Combination of Convolutional and Recursive Neural Networks. Neural Process Lett 50, 2745–2761 (2019). https://doi.org/10.1007/s11063-019-10049-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-019-10049-1

Keywords

Search

Navigation