Joint Learning of Entity Semantics and Relation Pattern for Relation Extraction

  • Suncong Zheng
  • Jiaming Xu
  • Hongyun Bao
  • Zhenyu Qi
  • Jie Zhang
  • Hongwei Hao
  • Bo Xu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9851)


Relation extraction is identifying the relationship of two given entities in the text. It is an important step in the task of knowledge extraction, which plays a vital role in automatic construction of knowledge base. When extracting entities’ relations from sentences, some keywords can reflect the relation pattern, besides, the semantic properties of given entities can also help to distinguish some confusing relations. Based on the above observations, we propose a mixture convolutional neural network for the task of relation extraction, which can simultaneously learn the semantic properties of entities and the keyword information related to the relation. We conduct experiments on the SemEval-2010 Task 8 dataset. The method we propose achieves the state-of-the-art result without using any external information. Additionally, the experimental results also show that our approach can learn the semantic relationship of the given entities effectively.


Relation extraction Convolutional neural network Entity embedding Keywords extraction 



This work is also supported by the National High Technology Research and Development Program of China (863 Program) (Grant No. 2015AA015402), the Hundred Talents Program of Chinese Academy of Sciences (No. Y3S4011D31), the NSFC project (No. 61501463) and National Natural Science Foundation (Grant No. 71402178).


  1. 1.
    Rink, B., et al.: UTD: Classifying semantic relations by combining lexical and semantic resources. In: 5th SE, pp. 256–259 (2010)Google Scholar
  2. 2.
    Kambhatla, N.: Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In: 43th ACL, pp. 22–26 (2004)Google Scholar
  3. 3.
    Zeng, D., et al.: Relation classification via convolutional deep neural network. In: 25th COLING, pp. 2335–2344 (2014)Google Scholar
  4. 4.
    dos Santos, C., Nogueira, et al.: Classifying relations by ranking with convolutional neural networks. In: 53th ACL, pp. 626–634 (2015)Google Scholar
  5. 5.
    Xu, Y., et al.: Classifying relations via long short term memory networks along shortest dependency paths. In: EMNLP (2015)Google Scholar
  6. 6.
    Yu, M., et al.: Factor-based compositional embedding models. In: NIPS Workshop on Learning Semantics (2014)Google Scholar
  7. 7.
    Socher, R., et al.: Semantic compositionality through recursive matrix-vector spaces. In: EMNLP, pp. 1201–1211 (2012)Google Scholar
  8. 8.
    Xu, K., et al.: Semantic relation classification via convolutional neural networks with simple negative sampling. In: EMNLP (2015)Google Scholar
  9. 9.
    Hendrickx, I., et al.: Semeval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: SE, pp. 94–99 (2009)Google Scholar
  10. 10.
    Sun, L., Han, X.: A Feature-Enriched Tree Kernel for Relation Extraction. In: the 52th ACL, pp. 61–67 (2014)Google Scholar
  11. 11.
    Blunsom, P., et al.: A convolutional neural network for modelling sentences. In: 52th ACL, (2014)Google Scholar
  12. 12.
    Kim, Y.: Convolutional neural networks for sentence classification. In: EMNLP, pp. 1746–1751 (2014)Google Scholar
  13. 13.
    Collobert, R., et al.: Natural language processing (almost) from scratch. In: JMLR, pp. 2493–2537 (2011)Google Scholar
  14. 14.
    Mikolov, T., et al.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp. 3111–3119 (2013)Google Scholar
  15. 15.
    Duan, K., et al.: Multi-category classification by soft-max combination of binary classifiers. J. Multiple Classifier Syst., 125–134 (2003)Google Scholar
  16. 16.
    Bottou, L.: Stochastic gradient learning in neural networks. J. Neuro-Nımes 9 (1991)Google Scholar
  17. 17.
    Fu, R., et al.: Learning semantic hierarchies via word embeddings. In: 52th ACL, pp. 1199–1209 (2014)Google Scholar
  18. 18.
    Wagstaff, K., et al.: Constrained k-means clustering with background knowledge. In: 18th ICML, pp. 577–584 (2001)Google Scholar
  19. 19.
    Chen, W.-Y., et al.: Parallel spectral clustering in distributed systems. J. TPAMI, 568–586 (2011)Google Scholar
  20. 20.
    Cai, D., et al.: Document clustering using locality preserving indexing. IEEE Trans. J. Knowl. Data Eng., 1624–1637 (2005)Google Scholar
  21. 21.
    Van der Maaten, L., et al.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)zbMATHGoogle Scholar
  22. 22.
    Jiaming, X., Peng, W., et al.: Short text clustering via convolutional neural networks. In: The NAACL, pp. 62–69 (2015)Google Scholar
  23. 23.
    Hearst, M.A., Dumais, et al.: Support vector machines. In: IEEE Intelligent Systems and their Applications, pp. 18–28 (1998)Google Scholar
  24. 24.
    Phillips, S.J., et al.: Maximum entropy modeling of species geographic distributions. In: Ecological modelling, pp. 231–259 (2006)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Suncong Zheng
    • 1
  • Jiaming Xu
    • 1
  • Hongyun Bao
    • 1
  • Zhenyu Qi
    • 1
  • Jie Zhang
    • 1
  • Hongwei Hao
    • 1
  • Bo Xu
    • 1
  1. 1.Institute of Automation, Chinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations