A Survey of Relation Extraction of Knowledge Graphs

  • Aoran Li
  • Xinmeng Wang
  • Wenhuan Wang
  • Anman Zhang
  • Bohan LiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11809)


With the widespread use of big data, knowledge graph has become a new hotspot. It is used in intelligent question answering, recommendation system, map navigation and so on. Constructing a knowledge graph includes ontology construction, annotated data, relation extraction, and ontology inspection. Relation extraction is to solve the problem of entity semantic linking, which is of great significance to many natural language processing applications. Research related to relation extraction has gained momentum in recent years, necessitating a comprehensive survey to offer a bird’s-eye view of the current state of relation extraction. In this paper, we discuss the development process of relation extraction, and classify the relation extraction algorithms in recent years. Furthermore, we discuss deep learning, reinforcement learning, active learning and transfer learning. By analyzing the basic principles of supervised learning, unsupervised learning, semi-supervised learning and distant supervision, we elucidate the characteristics of different relation extraction algorithms, and give the potential research directions in the future.


Knowledge graph Relation extraction Machine learning 


  1. 1.
    Xu, B., et al.: CN-DBpedia: a never-ending chinese knowledge extraction system. In: Benferhat, S., Tabia, K., Ali, M. (eds.) IEA/AIE 2017. LNCS (LNAI), vol. 10351, pp. 428–438. Springer, Cham (2017). Scholar
  2. 2.
    Niu, X., Sun, X.R., Wang, H.F., et al.: Zhishi.meweaving Chinese linking open data. In: Proceedings of the 10th International Semantic Web Conference, Bonn, Germany, pp. 205–220 (2011)Google Scholar
  3. 3.
    Pan, J.Z., Horrocks, I.: RDFS(FA): connecting RDF(S) and OWL DL. IEEE Trans. Knowl. Data Eng. 19(2), 192–206 (2007).
  4. 4.
    Mcguiness, D.L., Harmelen, F.: OWL Web ontology language overview. W3C Recomm. 63(45), 990–996 (2004)Google Scholar
  5. 5.
    Qiao, L., Yang, L., Hong, D., et al.: Knowledge graph construction techniques. J. Comput. Res. Dev. 53(3), 582–600 (2016). (in Chinese)Google Scholar
  6. 6.
    Zhang, C., Chang, L., Wang, W., Chen, H., Bin, C.: Question and answer over fine-grained knowledge graph based on BiLSTM-CRF (2019)Google Scholar
  7. 7.
    Proceedings of the 6th Message Understanding Conference (MUC - 7). National Institute of Standars and Technology (1998)Google Scholar
  8. 8.
    Suchanek, F.M., Kasneci, G., Weikum, G.: YAGO: a core of semantic knowledge unifying WordNet and Wikipedia. In: Proceedings of WWW (2007)Google Scholar
  9. 9.
    Banko, M., Cafarella, M.J., Soderland, S., et al.: Open information extraction for the web. In: Proceedings of the 20th Int Joint Conf on Artificial Intelligence, pp. 2670–2676. ACM, New York (2007)Google Scholar
  10. 10.
    Yang, B., Cai, D.-F., Yang, H.: Progress in open information extraction. J. Chin. Inf. Process. 4, 1–11 (2014)Google Scholar
  11. 11.
    Etzioni, O., Cafarella, M., Downey, D., et al.: Unsupervised named-entity extraction from the web: an experimental study. Artif. Intell. 165(1), 91–134 (2005)CrossRefGoogle Scholar
  12. 12.
    Banko, M., Cafarella, M.J., Soderland, S., et al.: Open information extraction from the web. In: Proceedings of IJCAI (2007)Google Scholar
  13. 13.
    Banko, M., Etzioni, O.: The tradeoffs between open and traditional relation extraction. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (2008)Google Scholar
  14. 14.
    Wu, F., Weld, D.S.: Open information extraction using Wikipedia. In: Proceedings of Annual Meeting of the Association for Computational Linguistics, pp. 118–127 (2010)Google Scholar
  15. 15.
    Fader, A., Soderland, S., Etzioni, O.: Identifying relations for open information extraction. In: Proceedings of Conference on Empirical Methods in Natural Language Processing (2011)Google Scholar
  16. 16.
    Etzioni, O., Fader, A., Christensen, J., et al.: Open information extraction: the second generation. In: Proceedings of International Joint Conference on Artificial Intelligence (2011)Google Scholar
  17. 17.
    Xu, J., Zhang, Z., Wu, Z.: Review on techniques of entity relation extraction. New Technol. Libr. Inf. Serv. 168(8), 18–23 (2008)Google Scholar
  18. 18.
    Gudovskiy, D., Hodgkinson, A.: Explanation-based attention for semi-supervised deep active learning (2019)Google Scholar
  19. 19.
    Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M.: Dueling network architectures for deep reinforcement learning (2019)Google Scholar
  20. 20.
    Hasegawa, T., Sekine, S., Grishman, R.: Discovering relations among named entities from large corpora. In: Proceedings of ACL-2004, pp. 415–422 (2004)Google Scholar
  21. 21.
    Socher, R., Chen, D., Manning, C.D., Ng, A.Y.: Reasoning with neural tensor networks for knowledge base completion (2013)Google Scholar
  22. 22.
    Bordes, A., Weston, J., Collobert, R., Bengio, Y.: Learning structured embeddings of knowledge bases. In: AAAI (2011)Google Scholar
  23. 23.
    Jenatton, R., Le Roux, N., Bordes, A., Obozinski, G.: A latent factor model for highly multi-relational data. In: NIPS (2012)Google Scholar
  24. 24.
    Heck, L., Hakkani-Tür, D., Tur, G.: Leveraging knowledge graphs for web-scale unsupervised semantic parsing. In: ISCA (2013)Google Scholar
  25. 25.
    Luus, F., Khan, N., Akhalwaya, I.: Active learning with TensorBoard projector (2019)Google Scholar
  26. 26.
    Liu, F., Zhong, Z., Lei, L., Wu, Y.: Entity relation extraction method based on machine learning (2013)Google Scholar
  27. 27.
    Xia, S., Lehong, D.: Feature-based approach to Chinese term relation extraction In: 2009 International Conference on Signal Processing Systems, pp. 410–414 (2009)Google Scholar
  28. 28.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge University (2000)Google Scholar
  29. 29.
    Zhang, T.: Regularized winnow methods. In: Advances in Neural Information Processing Systems 13, pp. 703–709 (2001)Google Scholar
  30. 30.
    Dong, X.L., Gabrilovich, E., Heitz, G.: Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: KDD (2014)Google Scholar
  31. 31.
    Zhou, Z.-H.: Cooperative Training Style in Semi-Supervised Learning. Machine Learning and Its Applications, pp. 259–275. Tsinghua University Press, Beijing (2007)Google Scholar
  32. 32.
    Chapelle, O., Schölkopf, B., Zien, A. (eds.): Semi-Supervised Learning (2006). The MIT Press, CambridgeGoogle Scholar
  33. 33.
    Pise, N.N., Kulkarni, P.: A survey of semi-supervised learning methods. In: 2008 International Conference on Computational Intelligence and Security (2008)Google Scholar
  34. 34.
    Zhou, Z.-H., Li, M.: Tri-training: exploiting unlabeled data using three classifiers. IEEE Trans. Knowl. Data Eng. 17(11), 1529–1541 (2005)CrossRefGoogle Scholar
  35. 35.
    Li, M., Zhou, Z.-H.: Improve computer-aided diagnosis with machine learning techniques using undiagnosed samples. IEEE Trans. Syst. 19(11), 1479–1493 (2007)Google Scholar
  36. 36.
    Zhang, M.-L., Zhou, Z.-H.: CoTRADE: confident co-training with data editing. IEEE Trans. Syst. Man Cybern. Part B Cybern. 41, 1612–1626 (2011)CrossRefGoogle Scholar
  37. 37.
    Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., Weld, D.S.: CoTRADE: knowledge-based weak supervision for information extraction of overlapping relations. In: The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 541–550 (2011)Google Scholar
  38. 38.
    Li, Q., Han, Z., Wu, X.-M.: CoTRADE: deeper insights into graph convolutional networks for semi-supervised learning. In: The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) (2018)Google Scholar
  39. 39.
    Luan, Y., Wadden, D., He, L., Shah, A., Ostendorf, M., Hajishirzi, H.: CoTRADE: a general framework for information extraction using dynamic span graphs. In: NAACL (2019)Google Scholar
  40. 40.
    Agrawal, K., Mittal, A., Pudi, V.: CoTRADE: scalable, semi-supervised extraction of structured information from scientific literature, pp. 11–20. Association for Computational Linguistics (2019)Google Scholar
  41. 41.
    Kim, S.N., Medelyan, O., Kan, M.-Y., Baldwin, T.: SemEval-2010 task 5: automatic keyphrase extraction from scientifific articles. In: Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval 2010, Stroudsburg, PA, USA, pp. 21–26. Association for Computational Linguistics (2010)Google Scholar
  42. 42.
    Gollapalli, S.D., Caragea, C.: Extracting keyphrases from research papers using citation networks. In: Proceedings of the Twenty-Eighth AAAI Conference on Artifificial Intelligence, AAAI 2014, pp. 1629–1635. AAAI Press (2014)Google Scholar
  43. 43.
    Jaidka, K., Chandrasekaran, M.K., Rustagi, S., Kan, M.-Y.: Insights from CL-SciSumm 2016: the faceted scientific document summarization shared task. Int. J. Digit. Libr. 19(2), 163–171 (2016)Google Scholar
  44. 44.
    Agrawal, K., Mittal, A., Pudi, V.: Scalable, semi-supervised extraction of structured information from scientifific literature (2019)Google Scholar
  45. 45.
    Drugman, T., Pylkkonen, J., Kneser, R.: Active and semi-supervised learning in ASR: benefits on the acoustic and language models (2019)Google Scholar
  46. 46.
    Arora, C., Sabetzadeh, M., Nejati, S., Briand, L.: An active learning approach for improving the accuracy of automated domain model extraction (2019)Google Scholar
  47. 47.
    Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data, ACL 2009 (2009)Google Scholar
  48. 48.
    Surdeanu, M., Tibshirani, J., Nallapati, R., Manning, C.D.: Multi-instance multi-label learning for relation extraction. In: Proceedings of EMNLP-CoNLL, pp. 455–465 (2012)Google Scholar
  49. 49.
    Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: Proceedings of COLING, pp. 2335–2344 (2014)Google Scholar
  50. 50.
    Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks (2015)Google Scholar
  51. 51.
    Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances (2016)Google Scholar
  52. 52.
    Wang, G., Zhang, W., Wang, R., Zhou, Y.: Label-free distant supervision for relation extraction via knowledge graph embedding (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Aoran Li
    • 1
  • Xinmeng Wang
    • 1
  • Wenhuan Wang
    • 1
  • Anman Zhang
  • Bohan Li
    • 1
    • 2
    Email author
  1. 1.College of Computer Science and TechnologyNanjing University of Aeronautics and AstronauticsNanjingChina
  2. 2.Collaborative Innovation Center of Novel Software Technology and IndustrializationNanjingChina

Personalised recommendations