Learning Attention-Based Translational Knowledge Graph Embedding via Nonlinear Dynamic Mapping

Abstract

Knowledge graph embedding has become a promising method for knowledge graph completion. It aims to learn low-dimensional embeddings in continuous vector space for each entity and relation. It remains challenging to learn accurate embeddings for complex multi-relational facts. In this paper, we propose a new translation-based embedding method named ATransD-NL to address the following two observations. First, most existing translational methods do not consider contextual information that have been proved useful for improving performance of link prediction. Our method learns attention-based embeddings for each triplet taking into account influence of one-hop or potentially multi-hop neighbourhood entities. Second, we apply nonlinear dynamic projection of head and tail entities to relational space, to capture nonlinear correlations among entities and relations due to complex multi-relational facts. As an extension of TransD, our model only introduces one more extra parameter, giving a good tradeoff between model complexity and the state-of-the-art predictive accuracy. Compared with state-of-the-art translation-based methods and the neural-network based methods, experiment results show that our method delivers substantial improvements over baselines on the MeanRank metric of link prediction, e.g., an improvement of 35.6% over the attention-based graph embedding method KBGAT and an improvement of 64% over the translational method TransMS on WN18 database, with comparable performance on the Hits@10 metric.

Z. Wang and H. Xu are co-first authors. This work was supported in part by the Ministry of Science and Technology of China under Grant No. 2018YFC0830400, National Natural Science Foundation of China under Grant No. 61802126, 61832015, 62072176, the Inria-CAS joint project Quasar and Shanghai Pujiang Program under Grant No. 17PJ1402200.