Knowledge Graph Embedding with Order Information of Triplets

  • Jun Yuan
  • Neng Gao
  • Ji XiangEmail author
  • Chenyang Tu
  • Jingquan Ge
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11441)


Knowledge graphs (KGs) are large scale multi-relational directed graph, which comprise a large amount of triplets. Embedding knowledge graphs into continuous vector space is an essential problem in knowledge extraction. Many existing knowledge graph embedding methods focus on learning rich features from entities and relations with increasingly complex feature engineering. However, they pay little attention on the order information of triplets. As a result, current methods could not capture the inherent directional property of KGs fully. In this paper, we explore knowledge graphs embedding from an ingenious perspective, viewing a triplet as a fixed length sequence. Based on this idea, we propose a novel recurrent knowledge graph embedding method RKGE. It uses an order keeping concatenate operation and a shared sigmoid layer to capture order information and discriminate fine-grained relation-related information. We evaluate our method on knowledge graph completion on benchmark data sets. Extensive experiments show that our approach outperforms state-of-the-art baselines significantly with relatively much lower space complexity. Especially on sparse KGs, RKGE achieves a 86.5% improvement at Hits@1 on FB15K-237. The outstanding results demonstrate that the order information of triplets is highly beneficial for knowledge graph embedding.


Knowledge graph Embedding Order information Recurrent model 



This work is supported by the National Key Research and Development Program of China, and National Natural Science Foundation of China (No. U163620068).


  1. 1.
    Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of SIGMOD, pp. 1247–1250 (2008)Google Scholar
  2. 2.
    Bordes, A., Usunier, N., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Proceedings of NIPS, pp. 2787–2795 (2013)Google Scholar
  3. 3.
    Cai, L., Wang, W.Y.: KBGAN: adversarial learning for knowledge graph embeddings. In: Proceedings of NAACL (2018)Google Scholar
  4. 4.
    Dai, Q.N., Tu, D.N., Nguyen, D.Q., Phung, D.: A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of NAACL (2018)Google Scholar
  5. 5.
    Dettmers, T., Pasquale, M., Pontus, S., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of AAAI, pp. 1811–1818 (2018)Google Scholar
  6. 6.
    Dong, X., et al.: Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: Proceedings of SIGKDD, pp. 601–610 (2014)Google Scholar
  7. 7.
    Garciaduran, A., Bordes, A., Usunier, N.: Composing relationships with translations. In: Proceedings of EMNLP, pp. 286–290 (2015)Google Scholar
  8. 8.
    Garciaduran, A., Niepert, M.: KBLRN: end-to-end learning of knowledge base representations with latent, relational, and numerical features. In: Proceedings of UAI (2017)Google Scholar
  9. 9.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  10. 10.
    Guo, S., Wang, Q., Wang, L., Wang, B., Guo, L.: Knowledge graph embedding with iterative guidance from soft rules. In: Proceedings of AAAI (2018)Google Scholar
  11. 11.
    Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of ACL, pp. 687–696 (2015)Google Scholar
  12. 12.
    Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Proceedings of AAAI, pp. 985–991 (2016)Google Scholar
  13. 13.
    Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. In: Proceedings of EMNLP, pp. 705–714 (2015)Google Scholar
  14. 14.
    Lin, Y., Liu, Z., Zhu, X., Zhu, X., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of AAAI, pp. 2181–2187 (2015)Google Scholar
  15. 15.
    Lipton, Z.C.: A critical review of recurrent neural networks for sequence learning. CoRR abs/1506.00019 (2015).
  16. 16.
    Liu, H., Wu, Y., Yang, Y.: Analogical inference for multi-relational embeddings. In: Proceedings of ICML (2017)Google Scholar
  17. 17.
    Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)CrossRefGoogle Scholar
  18. 18.
    Qian, W., Fu, C., Zhu, Y., Cai, D., He, X.: Translating embeddings for knowledge graph completion with relation attention mechanism. In: Proceedings of IJCAI, pp. 4286–4292 (2018)Google Scholar
  19. 19.
    Socher, R., Chen, D., Manning, C.D., Ng, A.Y.: Reasoning with neural tensor networks for knowledge base completion. In: Proceedings of NIPS, pp. 926–934 (2013)Google Scholar
  20. 20.
    Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of EMNLP, pp. 1631–1642 (2013)Google Scholar
  21. 21.
    Toutanova, K., Chen, D.: Observed versus latent features for knowledge base and text inference. In: Proceedings of CVCS (2015)Google Scholar
  22. 22.
    Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of ICML, vol. 48, pp. 2071–2080 (2016)Google Scholar
  23. 23.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of AAAI, pp. 1112–1119 (2014)Google Scholar
  24. 24.
    Xu, H., Yankai, L., Ruobing, X., Zhiyuan, L., Maosong, S.: OpenKE: an open-source framework for knowledge embedding (2017).
  25. 25.
    Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of ICLR (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Jun Yuan
    • 1
    • 2
  • Neng Gao
    • 1
  • Ji Xiang
    • 1
    Email author
  • Chenyang Tu
    • 1
  • Jingquan Ge
    • 1
    • 2
  1. 1.Institute of Information EngineeringChinese Academy of SciencesBeijingChina
  2. 2.School of Cyber SecurityUniversity of Chinese Academy of SciencesBeijingChina

Personalised recommendations