Abstract
Relation Extraction (RE) is a premier task of information extraction (IE) and crucial to many applications including knowledge graph completion (KGC). In recent years, some RE models have employed the topic knowledge of relations through topic words to enrich relation representations, demonstrating better performance than traditional distantly supervised paradigms. However, these models have not taken different syntactic information of relations into account, which have been proven significant in many NLP tasks. In this paper, we propose a novel RE pipeline which incorporates syntactic information into relation representations to enhance RE performance. Representations of sentence and relation in our pipeline are generated by a modified multi-head self-attention structure respectively, where the sentence is represented based on its words and the relation is represented based on the relation-specific embeddings of its topic words. Furthermore, all sentences labeled with the input relation are used to construct an entire weighted directed graph based on their dependency trees. Then, the relation-specific embeddings of words (nodes) in the graph are learned by a GCN-based model. Our extensive experiments have justified that our pipeline significantly outperforms other RE models thanks to the incorporation of syntactic information.
This work is supported by Shanghai Science and technology innovation action plan (No. 19511120400).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alexandros, K., Suresh, M.: Dependency based embeddings for sentence classification tasks. In: Proceedings of NAACL (2016)
Anita, A., Anna, C.: Barrier features for classification of semantic relations. In: Proceedings of RANLP (2011)
Ashish, V., Noam, S., Niki, P., et al.: Attention is all you need. In: Proceedings of NIPS (2017)
Bastings, J., Titov, I., Aziz, W., et al.: Graph convolutional encoders for syntax-aware neural machine translation. In: Proceedings of EMNLP (2017)
Bunescu, R.C., Mooney, R.J.: A shortest path dependency kernel for relation extraction. In: Proceedings of HLT/EMNLP (2005)
Chen, L., Jianxin, L., Yangqiu, S., Ziwei, L.: Training and evaluating improved dependency-based word embeddings. In: Proceedings of AAAI (2018)
Christopher, M., Mihai, S., John, B., Jenny, F., Steven, B., David, M.: The stanford coreNLP natural language processing toolkit. In: Proceedings of ACL (2014)
Feng, J., Huang, M., Zhao, L., Yang, Y., Zhu, X.: Reinforcement learning for relation classification from noisy data. In: Proceedings of AAAI (2018)
GuoDong, Z., Jian, S., Jie, Z., Min, Z.: Exploring various knowledge in relation extraction. In: Proceedings of ACL (2005)
Ji, G., Liu, K., He, S., Zhao, J.: Distant supervision for relation extraction with sentence-level attention and entity descriptions. In: Proceedings of AAAI (2017)
Jiang, H., Cui, L., Xu, Z., et al.: Relation extraction using supervision from topic knowledge of relation labels (2019)
Marcheggiani, D., Titov, I.: Encoding sentences with graph convolutional networks for semantic role labeling. In: Proceedings of EMNLP (2017)
Matt, K., Yu, S., Nicholas, K., Kilian, W.: From word embeddings to document distances. In: Proceedings of ICML (2015)
Michaël, D., Xavier, B., Pierre, V.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of NIPS (2016)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013)
Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings of ACL/AFNLP (2009)
N, K.T., Max, W.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Omer, L., Yoav, G.: Dependency-based word embeddings. In: Proceedings of ACL (2014)
Riedel, S., Yao, L., McCallum, A.: Modeling relations and their mentions without labeled text. In: Proceedings of ECML/PKDD (2010)
Rocktäschel, T., Singh, S., Riedel, S.: Injecting logical background knowledge into embeddings for relation extraction. In: Proceedings of NAACL (2015)
Shikhar, V., Manik, B., et al.: Incorporating syntactic and semantic information in word embeddings using graph convolutional networks. In: Proceedings of ACL (2019)
Vashishth, S., Dasgupta, S.S., Ray, S.N., Talukdar, P.: Dating documents using graph convolution networks. In: Proceedings of ACL (2018)
Wang, C., Fan, J., Kalyanpur, A., Gondek, D.: Relation extraction with relation topics. In: Proceedings of EMNLP (2011)
Yankai, L., Shiqi, S., et al.: Neural relation extraction with selective attention over instances. In: Proceedings of ACL, pp. 2124–2133 (2016)
Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of AAAI (2019)
Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of EMNLP (2015)
Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network, pp. 2335–2344 (2014)
Zhang, D., Wang, D.: Relation classification via recurrent neural network. arXiv preprint arXiv:1508.01006 (2015)
Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., Xu, B.: Joint extraction of entities and relations based on a novel tagging scheme (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cui, L., Yang, D., Cheng, J., Xiao, Y. (2021). Incorporating Syntactic Information into Relation Representations for Enhanced Relation Extraction. In: Karlapalem, K., et al. Advances in Knowledge Discovery and Data Mining. PAKDD 2021. Lecture Notes in Computer Science(), vol 12714. Springer, Cham. https://doi.org/10.1007/978-3-030-75768-7_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-75768-7_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-75767-0
Online ISBN: 978-3-030-75768-7
eBook Packages: Computer ScienceComputer Science (R0)