Advertisement

Domain-Aware Sentiment Classification with GRUs and CNNs

  • Guangyuan PiaoEmail author
  • John G. Breslin
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 927)

Abstract

In this paper, we describe a deep neural network architecture for domain-aware sentiment classification task with the purpose of the sentiment classification of product reviews in different domains and evaluating nine pre-trained embeddings provided by the semantic sentiment classification challenge at the 15th Extended Semantic Web Conference. The proposed approach combines the domain and the sequence of word embeddings of the summary or text of each review for Gated Recurrent Units (GRUs) to produce the corresponding sequence of embeddings by being aware of the domain and previous words. Afterwards, it extracts local features using Convolutional Neural Networks (CNNs) from the output of the GRU layer. The two sets of local features extracted from the domain-aware summary and text of a review are concatenated into a single vector, and are used for classifying the sentiment of a review. Our approach obtained 0.9643 F1-score on the test set and achieved the 1st place in the first task of the Semantic Sentiment Analysis Challenge at the 15th Extended Semantic Web Conference.

Notes

Acknowledgments

This publication has emanated from research conducted with the financial support of Science Foundation Ireland (SFI) under Grant Number SFI/12/RC/2289 (Insight Centre for Data Analytics).

References

  1. 1.
    Atzeni, Mattia, Dridi, Amna, Reforgiato Recupero, Diego: Fine-grained sentiment analysis on financial microblogs and news headlines. In: Dragoni, Mauro, Solanki, Monika, Blomqvist, Eva (eds.) SemWebEval 2017. CCIS, vol. 769, pp. 124–128. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-69146-6_11CrossRefGoogle Scholar
  2. 2.
    Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
  3. 3.
    Cliche, M.: BB\_twtr at SemEval-2017 Task 4: twitter Sentiment Analysis with CNNs and LSTMs. CoRR abs/1704.0 (2017), arXiv:1704.06125
  4. 4.
    Cortis, K., et al.: Semeval-2017 task 5: fine-grained sentiment analysis on financial microblogs and news. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 519–535 (2017)Google Scholar
  5. 5.
    Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)CrossRefGoogle Scholar
  6. 6.
    Federici, Marco, Dragoni, Mauro: A knowledge-based approach for aspect-based opinion mining. In: Sack, Harald, Dietze, Stefan, Tordai, Anna, Lange, Christoph (eds.) SemWebEval 2016. CCIS, vol. 641, pp. 141–152. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46565-4_11CrossRefGoogle Scholar
  7. 7.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997).  https://doi.org/10.1162/neco.1997.9.8.1735CrossRefGoogle Scholar
  8. 8.
    Iguider, Walid, Reforgiato Recupero, Diego: Language independent sentiment analysis of the shukran social network using apache spark. In: Dragoni, Mauro, Solanki, Monika, Blomqvist, Eva (eds.) SemWebEval 2017. CCIS, vol. 769, pp. 129–132. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-69146-6_12CrossRefGoogle Scholar
  9. 9.
    Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: The 52nd Annual Meeting of the Association for Computational Linguistics (2014)Google Scholar
  10. 10.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Conference on Empirical Methods on Natural Language Processing (2014)Google Scholar
  11. 11.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  12. 12.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  13. 13.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  14. 14.
    LeCun, Y., et al.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems, pp. 396–404 (1990)Google Scholar
  15. 15.
    Petrucci, Giulio, Dragoni, Mauro: The IRMUDOSA system at ESWC-2017 challenge on semantic sentiment analysis. In: Dragoni, Mauro, Solanki, Monika, Blomqvist, Eva (eds.) SemWebEval 2017. CCIS, vol. 769, pp. 148–165. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-69146-6_14CrossRefGoogle Scholar
  16. 16.
    Piao, G., Breslin, J.G.: Financial aspect and sentiment predictions with deep neural networks: an ensemble approach. In: Financial Opinion Mining and Question Answering Workshop at The Web Conference (WWW). ACM (2018)Google Scholar
  17. 17.
    Reforgiato Recupero, Diego, Cambria, Erik, Di Rosa, Emanuele: Semantic sentiment analysis challenge at ESWC2017. In: Dragoni, Mauro, Solanki, Monika, Blomqvist, Eva (eds.) SemWebEval 2017. CCIS, vol. 769, pp. 109–123. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-69146-6_10CrossRefGoogle Scholar
  18. 18.
    Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Zhang, S., Yao, L., Sun, A.: Deep learning based recommender system: a survey and new perspectives. CoRR abs/1707.0 (2017), arXiv:1707.07435
  20. 20.
    Zhang, Y., Wallace, B.C.: A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. CoRR abs/1510.0 (2015), arXiv:1510.03820

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Insight Centre for Data Analytics, Data Science Institute, National University of Ireland GalwayGalwayIreland
  2. 2.IDA Business ParkGalwayIreland

Personalised recommendations