Advertisement

Attention-Based Combination of CNN and RNN for Relation Classification

  • Xiaoyu Guo
  • Hui Zhang
  • Rui Liu
  • Xin Ding
  • Runqi Tian
  • Bencheng Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11304)

Abstract

Relation classification is an essential task in natural language processing (NLP) in order to extract structured data from sentences. In this paper, we propose a novel model Att-ComNN combining convolutional neural network (CNN) and bidirectional recurrent neural network (RNN) for relation classification. By combining RNN and CNN, we obtain more accurate context representations of words, which benefits classifying relations. Besides, with both shortest dependency path (SDP) attention and pooling attention added, this model captures the most informative context representation for better classification without using other handcrafted features. The results of experiments show that our model improves the relation classification performance on the SemEval-2010 Task 8 and outperforms most of previous state-of-the-art methods, including those depending on much richer forms of handcrafted features and prior knowledge.

Keywords

Relation classification Deep neural network Attention mechanism 

Notes

Acknowledgements

This work is supported by National Key R&D Program of China (No. 2017YFB1400200).

References

  1. 1.
    Bunescu, R.C., Mooney, R.J.: A shortest path dependency kernel for relation extraction. In: Proceedings of the Conference Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, HLT/EMNLP 2005, Vancouver, British Columbia, Canada, 6–8 October 2005, pp. 724–731 (2005)Google Scholar
  2. 2.
    Cai, R., Zhang, X., Wang, H.: Bidirectional recurrent convolutional neural network for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Volume 1: Long Papers, Berlin, Germany, 7–12 August 2016 (2016)Google Scholar
  3. 3.
    Chen, J., Ji, D., Tan, C.L., Niu, Z.: Unsupervised feature selection for relation extraction. In: Second International Joint Conference on Natural Language Processing - IJCNLP 2005 - Companion Volume to the Proceedings of Conference including Posters/Demos and tutorial abstracts, Jeju Island, Republic of Korea, 11–13 October 2005 (2005)Google Scholar
  4. 4.
    Cho, K., van Merrienboer, B., Gülçehre, Ç., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. CoRR abs/1406.1078 (2014)Google Scholar
  5. 5.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, Chia Laguna Resort, Sardinia, Italy, 13–15 May 2010, pp. 249–256 (2010)Google Scholar
  6. 6.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013)Google Scholar
  7. 7.
    Rink, B., Harabagiu, S.M.: UTD: classifying semantic relations by combining lexical and semantic resources. In: Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval@ACL 2010, Uppsala University, Uppsala, Sweden, 15–16 July 2010, pp. 256–259 (2010)Google Scholar
  8. 8.
    Rotsztejn, J., Hollenstein, N., Zhang, C.: Eth-ds3lab at semeval-2018 task 7: effectively combining recurrent and convolutional neural networks for relation classification and extraction. CoRR abs/1804.02042 (2018)Google Scholar
  9. 9.
    dos Santos, C.N., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. CoRR abs/1504.06580 (2015)Google Scholar
  10. 10.
    Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL 2012, Jeju Island, Korea, 12–14 July 2012, pp. 1201–1211 (2012)Google Scholar
  11. 11.
    Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., Manning, C.D.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, A meeting of SIGDAT, a Special Interest Group of the ACL, John McIntyre Conference Centre, Edinburgh, UK, 27–31 July 2011, pp. 151–161 (2011)Google Scholar
  12. 12.
    Wang, L., Cao, Z., de Melo, G., Liu, Z.: Relation classification via multi-level attention CNNs. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Volume 1: Long Papers, Berlin, Germany, 7–12 August 2016 (2016)Google Scholar
  13. 13.
    Xu, K., Feng, Y., Huang, S., Zhao, D.: Semantic relation classification via convolutional neural networks with simple negative sampling. CoRR abs/1506.07650 (2015)Google Scholar
  14. 14.
    Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., Jin, Z.: Classifying relations via long short term memory networks along shortest dependency paths. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, 17–21 September 2015, pp. 1785–1794 (2015)Google Scholar
  15. 15.
    Yu, J., Jiang, J.: Pairwise relation classification with mirror instances and a combined convolutional neural network. In: COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, Osaka, Japan, 11–16 December 2016, pp. 2366–2377 (2016)Google Scholar
  16. 16.
    Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: COLING 2014, 25th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, Dublin, Ireland, 23–29 August 2014, pp. 2335–2344 (2014)Google Scholar
  17. 17.
    Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., Xu, B.: Joint extraction of entities and relations based on a novel tagging scheme. CoRR abs/1706.05075 (2017)Google Scholar
  18. 18.
    Zhou, P., Shi, W., Tian, J., Qi, Z., Li, B., Hao, H., Xu, B.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Volume 2: Short Papers, Berlin, Germany, 7–12 August 2016Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Xiaoyu Guo
    • 1
  • Hui Zhang
    • 1
    • 2
  • Rui Liu
    • 1
  • Xin Ding
    • 1
  • Runqi Tian
    • 1
  • Bencheng Wang
    • 1
  1. 1.State Key Laboratory of Software Development Environment, School of Computer Science and EngineeringBeihang UniversityBeijingChina
  2. 2.Beijing Advanced Innovation Center for Big Data and Brain ComputingBeihang UniversityBeijingChina

Personalised recommendations