Skip to main content
SpringerLink
Log in
Book cover

International Conference on Knowledge Science, Engineering and Management

KSEM 2018: Knowledge Science, Engineering and Management pp 67–78Cite as

Attention Aware Bidirectional Gated Recurrent Unit Based Framework for Sentiment Analysis

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11061))

Abstract

Sentiment analysis is an effective technique and widely employed to analyze sentiment polarity of reviews and comments on the Internet. A lot of advanced methods have been developed to solve this task. In this paper, we propose an attention aware bidirectional GRU (Bi-GRU) framework to classify the sentiment polarity from the aspects of sentential-sequence modeling and word-feature seizing. It is composed of a pre-attention Bi-GRU to incorporate the complicated interaction between words by sentence modeling, and an attention layer to capture the keywords for sentiment apprehension. Afterward, a post-attention GRU is added to imitate the function of decoder, aiming to extract the predicted features conditioned on the above parts. Experimental study on commonly used datasets has demonstrated the proposed framework’s potential for sentiment classification.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The source code is available at https://github.com/MercuryTian/Attention-Aware-Bi-directional-GRU-Based-Framework-for-Sentiment-Analysis.git.

  2. 2.

    http://ai.stanford.edu/~amaas/data/sentiment/.

  3. 3.

    http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html.

  4. 4.

    http://www.cs.cornell.edu/people/pabo/movie-review-data/.

  5. 5.

    http://alt.qcri.org/semeval2014/task4/.

  6. 6.

    http://nlp.stanford.edu/projects/glove/.

References

  1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2014)

    Google Scholar 

  2. Bengio, Y., Simard, P.Y., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (1994)

    Article  Google Scholar 

  3. Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: Encoder-decoder approaches. In: Proceedings of 8th Workshop on Syntax, Semantics and Structure in Statistical Translation, pp. 103–111 (2014)

    Google Scholar 

  4. Chung, J., Gülçehre, Ç., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR abs/1412.3555 (2014)

    Google Scholar 

  5. Ding, X., Liu, B., Yu, P.S.: A holistic lexicon-based approach to opinion mining. In: Proceedings of 2008 International Conference on Web Search and Web Data Mining, pp. 231–240 (2008)

    Google Scholar 

  6. Goodfellow, I., Bengio, Y., Courville, A., Bengio, Y.: Deep Learning, vol. 1. MIT press, Cambridge (2016)

    MATH  Google Scholar 

  7. He, L., Lee, K., Lewis, M., Zettlemoyer, L.: Deep semantic role labeling: what works and what’s next. In: Proceedings of 55th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 473–483 (2017)

    Google Scholar 

  8. Hermann, K.M., et al.: Teaching machines to read and comprehend. In: Proceedings of 2015 Annual Conference on Neural Information Processing Systems, pp. 1693–1701 (2015)

    Google Scholar 

  9. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  10. Kudo, T., Matsumoto, Y.: Chunking with support vector machines. In: Proceedings of 2nd Meeting of the North American Chapter of the Association for Computational Linguistics (2001)

    Google Scholar 

  11. Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of 18th International Conference on Machine Learning, pp. 282–289 (2001)

    Google Scholar 

  12. Liu, J., Rong, W., Tian, C., Gao, M., Xiong, Z.: Attention aware semi-supervised framework for sentiment analysis. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017. LNCS, vol. 10614, pp. 208–215. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68612-7_24

    Chapter  Google Scholar 

  13. Lu, Y., Salem, F.M.: Simplified gating in long short-term memory (LSTM) recurrent neural networks. CoRR abs/1701.03441 (2017)

    Google Scholar 

  14. van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    MATH  Google Scholar 

  15. Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K.: Recurrent models of visual attention. In: Proceedings of 2014 Annual Conference on Neural Information Processing Systems, pp. 2204–2212 (2014)

    Google Scholar 

  16. Mousa, A.E., Schuller, B.W.: Contextual bidirectional long short-term memory recurrent neural network language models: a generative approach to sentiment analysis. In: Proceedings of 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1023–1032 (2017)

    Google Scholar 

  17. Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? Sentiment classification using machine learning techniques. In: Proceedings of 2002 Conference on Empirical Methods in Natural Language Processing (2002)

    Google Scholar 

  18. Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: Proceedings of 30th International Conference on Machine Learning, pp. 1310–1318 (2013)

    Google Scholar 

  19. Qiu, L., Lei, Q., Zhang, Z.: Advanced sentiment classification of tibetan microblogs on smart campuses based on multi-feature fusion. IEEE Access 6, 17896–17904 (2018)

    Article  Google Scholar 

  20. Raza, K., Alam, M.: Recurrent neural network based hybrid model for reconstructing gene regulatory network. Comput. Biol. Chem. 64, 322–334 (2016)

    Article  Google Scholar 

  21. Schuller, B.W., Mousa, A.E., Vryniotis, V.: Sentiment analysis and opinion mining: on optimal parameters and performances. Wiley Interdisc. Rev. Data Min. Knowl. Disc. 5(5), 255–263 (2015)

    Article  Google Scholar 

  22. Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing, pp. 606–615 (2016)

    Google Scholar 

  23. Wang, Y., Pal, A.: Detecting emotions in social media: a constrained optimization approach. In: Proceedings of 24th International Joint Conference on Artificial Intelligence, pp. 996–1002 (2015)

    Google Scholar 

  24. Zhang, B., Xiong, D., Su, J.: A GRU-gated attention model for neural machine translation. CoRR abs/1704.08430 (2017)

    Google Scholar 

  25. Zhang, M., Zhou, Z.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)

    Article  Google Scholar 

  26. Zhao, J., Liu, K., Xu, L.: Sentiment analysis: mining opinions, sentiments, and emotions. Comput. Linguist. 42(3), 595–598 (2016)

    Article  Google Scholar 

  27. Zhou, D., Zhang, X., Zhou, Y., Zhao, Q., Geng, X.: Emotion distribution learning from texts. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 638–647 (2016)

    Google Scholar 

  28. Zhou, J., Xu, W.: End-to-end learning of semantic role labeling using recurrent neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pp. 1127–1137 (2015)

    Google Scholar 

Download references

Acknowledgments

This work was partially supported by the National Natural Science Foundation of China (No. 61332018), and the Fundamental Research Funds for the Central Universities.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Beihang University, Beijing, China

    Zhengxi Tian, Wenge Rong, Jingshuang Liu & Zhang Xiong

  2. Sino-French Engineer School, Beihang University, Beijing, China

    Libin Shi

Authors
  1. Zhengxi Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenge Rong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Libin Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jingshuang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhang Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenge Rong .

Editor information

Editors and Affiliations

  1. University of Bristol, Bristol, United Kingdom

    Weiru Liu

  2. Università di Trento, Povo, Italy

    Fausto Giunchiglia

  3. Jilin University, Changchun, China

    Bo Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tian, Z., Rong, W., Shi, L., Liu, J., Xiong, Z. (2018). Attention Aware Bidirectional Gated Recurrent Unit Based Framework for Sentiment Analysis. In: Liu, W., Giunchiglia, F., Yang, B. (eds) Knowledge Science, Engineering and Management. KSEM 2018. Lecture Notes in Computer Science(), vol 11061. Springer, Cham. https://doi.org/10.1007/978-3-319-99365-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-99365-2_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-99364-5

  • Online ISBN: 978-3-319-99365-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation