Connecting Targets to Tweets: Semantic Attention-Based Model for Target-Specific Stance Detection

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10569)

Abstract

Understanding what people say and really mean in tweets is still a wide open research question. In particular, understanding the stance of a tweet, which is determined not only by its content, but also by the given target, is a very recent research aim of the community. It still remains a challenge to construct a tweet’s vector representation with respect to the target, especially when the target is only implicitly mentioned, or not mentioned at all in the tweet. We believe that better performance can be obtained by incorporating the information of the target into the tweet’s vector representation. In this paper, we thus propose to embed a novel attention mechanism at the semantic level in the bi-directional GRU-CNN structure, which is more fine-grained than the existing token-level attention mechanism. This novel attention mechanism allows the model to automatically attend to useful semantic features of informative tokens in deciding the target-specific stance, which further results in a conditional vector representation of the tweet, with respect to the given target. We evaluate our proposed model on a recent, widely applied benchmark Stance Detection dataset from Twitter for the SemEval-2016 Task 6.A. Experimental results demonstrate that the proposed model substantially outperforms several strong baselines, which include the state-of-the-art token-level attention mechanism on bi-directional GRU outputs and the SVM classifier.

Keywords

Target-specific Stance Detection Text classification Neural network Attention mechanism 

Notes

Acknowledgments

This work was supported by The Alan Turing Institute under the EPSRC grant EP/N510129/1.

References

  1. 1.
    Augenstein, I., Rocktäschel, T., Vlachos, A., Bontcheva, K.: Stance detection with bidirectional conditional encoding. In: Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing, pp. 876–885. ACL (2016)Google Scholar
  2. 2.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Proceedings of 3rd International Conference on Learning Representations (2015)Google Scholar
  3. 3.
    Cho, K., van Merrienboer, B., Gülçehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1724–1734. ACL (2014)Google Scholar
  4. 4.
    Ebrahimi, J., Dou, D., Lowd, D.: A joint sentiment-target-stance model for stance classification in tweets. In: Proceedings of 26th International Conference on Computational Linguistics, pp. 2656–2665. ACL (2016)Google Scholar
  5. 5.
    Gal, Y., Ghahramani, Z.: A theoretically grounded application of dropout in recurrent neural networks. In: Proceedings of Advances in Neural Information Processing Systems, vol. 29, pp. 1019–1027 (2016)Google Scholar
  6. 6.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751. ACL (2014)Google Scholar
  7. 7.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of 3rd International Conference on Learning Representations: Poster Session (2015)Google Scholar
  8. 8.
    Mohammad, S.M., Kiritchenko, S., Sobhani, P., Zhu, X., Cherry, C.: SemEval-2016 task 6: detecting stance in tweets. In: Proceedings of 10th International Workshop on Semantic Evaluation (2016)Google Scholar
  9. 9.
    Mohammad, S.M., Sobhani, P., Kiritchenko, S.: Stance and sentiment in tweets. ACM Trans. Internet Technol. 17(3), 26 (2017)CrossRefGoogle Scholar
  10. 10.
    Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing, vol. 14, pp. 1532–1543. ACL (2014)Google Scholar
  11. 11.
    Ruder, S., Ghaffari, P., Breslin, J.G.: A hierarchical model of reviews for aspect-based sentiment analysis. In: Proceedings of 2016 Conference on Empirical Methods in Natural Language Processing, pp. 999–1005. ACL (2016)Google Scholar
  12. 12.
    Schouten, K., Baas, F., Bus, O., Osinga, A., van de Ven, N., van Loenhout, S., Vrolijk, L., Frasincar, F.: Aspect-based sentiment analysis using lexico-semantic patterns. In: Cellary, W., Mokbel, M.F., Wang, J., Wang, H., Zhou, R., Zhang, Y. (eds.) WISE 2016. LNCS, vol. 10042, pp. 35–42. Springer, Cham (2016). doi: 10.1007/978-3-319-48743-4_3 CrossRefGoogle Scholar
  13. 13.
    Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetMATHGoogle Scholar
  14. 14.
    Tan, M., Xiang, B., Zhou, B.: LSTM-based deep learning models for non-factoid answer selection. In: Proceedings of 4th International Conference on Learning Representations: Workshop Track (2016)Google Scholar
  15. 15.
    Tang, D., Qin, B., Feng, X., Liu, T.: Effective LSTMs for target-dependent sentiment classification. In: Proceedings of 26th International Conference on Computational Linguistics, pp. 3298–3307. ACL (2016)Google Scholar
  16. 16.
    Wei, W., Zhang, X., Liu, X., Chen, W., Wang, T.: pkudblab at SemEval-2016 task 6: a specific convolutional neural network system for effective stance detection. In: Proceedings of 10th International Workshop on Semantic Evaluation (2016)Google Scholar
  17. 17.
    Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A.C., Salakhutdinov, R., Zemel, R.S., Bengio, Y.: Show, attend and tell: neural image caption generation with visual attention. In: Proceedings of 32nd International Conference on Machine Learning, pp. 2048–2057. ACM (2015)Google Scholar
  18. 18.
    Yang, Z., Yang, D., Dyer, C., He, X., Smola, A.J., Hovy, E.H.: Hierarchical attention networks for document classification. In: Proceedings of 2016 Conference of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies, pp. 1480–1489. ACL (2016)Google Scholar
  19. 19.
    Zarrella, G., Marsh, A.: MITRE at SemEval-2016 task 6: transfer learning for stance detection. In: Proceedings of 10th International Workshop on Semantic Evaluation (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of WarwickCoventryUK
  2. 2.University of LiverpoolLiverpoolUK

Personalised recommendations