Attention-Based Gated Convolutional Neural Networks for Distant Supervised Relation Extraction

  • Xingya LiEmail author
  • Yufeng Chen
  • Jinan Xu
  • Yujie Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11856)


Distant supervision is an effective method to generate large-scale labeled data for relation extraction without expensive manual annotation, but it inevitably suffers from the wrong labeling problem, which would make the corpus much noisy. However, the existing research work mainly focuses on sentence-level noise filtering, without considering noisy words which widely exist inside sentences. In this paper, we propose an attention-based gated piecewise convolutional neural networks (AGPCNNs) for distant supervised relation extraction, which can effectively reduce word-level noise by selecting the inner-sentence features. On the one hand, we construct a piecewise convolutional neural network with gate mechanism to extract features that are related to relations. On the other hand, we employ a soft-label strategy to enable model to select important features automatically. Furthermore, we adopt an attention mechanism after the piecewise pooling layer to obtain high-level positive features for relation predicting. Experimental results show that our method can effectively filter word-level noise and outperforms all baseline systems significantly.


Relation extraction Distant supervision Gate mechanism Attention mechanism 



The authors are supported by the National Nature Science Foundation of China (Nos. 61473294, 61370130 and 61876198), the Fundamental Research Funds for the Central Universities (Nos. 2015JBM033), and the International Science and Technology Cooperation Program of China (No. K11F100010).


  1. Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 1003–1011 (2009)Google Scholar
  2. Riedel, S., Yao, L., McCallum, A.: Modeling relations and their mentions without labeled text. In: Machine Learning and Knowledge Discovery in Databases, pp. 148–163 (2010)CrossRefGoogle Scholar
  3. Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1–2), 31–71 (1997)CrossRefGoogle Scholar
  4. Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., Weld, D. S.: Knowledge-based weak supervision for information extraction of overlapping relations. In: Proceedings of ACL, pp. 541–550. Association for Computational Linguistics (2011)Google Scholar
  5. Surdeanu, M., Tibshirani, J., Nallapati, R., Manning, C.D.: Multi-instance multi-label learning for relation extraction. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 455–465. Association for Computational Linguistics (2012)Google Scholar
  6. Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of EMNLP, pp. 1753–1762 (2015)Google Scholar
  7. Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2124–2133 (2016)Google Scholar
  8. Ji, G., Liu, K., He, S., Xu, L., Zhao, J.: Distant supervision for relation extraction with sentence-level attention and entity descriptions. In: AAAI, pp. 3060–3066 (2017)Google Scholar
  9. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  10. Wu, W., Chen, Y., Xu, J., Zhang, Y.: Attention-Based Convolutional Neural Networks for Chinese Relation Extraction. In: Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, pp. 147–158. Springer, Cham (2018)Google Scholar
  11. Wang, G., Zhang, W., Wang, R., Zhou, Y., Chen, X., Zhang, W., Zhu, H., Chen, H.: Label-free distant supervision for relation extraction via knowledge graph embedding. In: Proceedings of EMNLP, pp. 2246–2255 (2018)Google Scholar
  12. Kalchbrenner, N., Espeholt, L., Simonyan, K., Oord, A., Graves, A., Kavukcuoglu, K.: Neural machine translation in linear time. arXiv preprint arXiv, 1610.10099 (2016)Google Scholar
  13. Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.: Convolutional sequence to sequence learning. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1243–1252. JMLR. org (2017)Google Scholar
  14. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv, 1301.3781 (2013)Google Scholar
  15. Liu, T., Zhang, X., Zhou, W., Jia, W.: Neural relation extraction via inner-sentence noise reduction and transfer learning. In: Proceedings of EMNLP, pp. 2195–2204 (2018)Google Scholar
  16. Feng, J., Huang, M., Zhao, L., Yang, Y., Zhu X.: Reinforcement learning for relation classification from noisy data. In: AAAI, pp. 5779–5786 (2018)Google Scholar
  17. Qin, P., Xu, W., Wang, W.Y.: Robust distant supervision relation extraction via deep reinforcement learning. In: Proceedings of ACL, pp. 2137–2147 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Beijing Jiaotong UniversityBeijingChina

Personalised recommendations