Abstract
Knowledge Graph Embedding (KGE) has attracted more and more attention and has been widely used in downstream AI tasks. Some proposed models learn the embeddings of Knowledge Graph (KG) into a low-dimensional continuous vector space by optimizing a customized loss function. However, these methods either ignore the polysemy of entities/relations in KG, or cannot model the uncertainty of them. Therefore, we propose KG2GM (Knowledge Graph to Gaussian Mixture), a density-based KGE method, for modeling the polysemy and uncertainty of KG simultaneously. Each entity/relation in KG is represented by a Gaussian mixture probability distribution. Each Gaussian component of the mixed distribution represents one kind of semantics, where the mean vector denotes its space position, and the covariance matrix denotes its uncertainty. In this way we can model the polysemy and uncertainty of KG simultaneously. We employ symmetrized KL divergence and EL (Expected Likelihood) to measure the component similarities and make their probabilistic combination as the score function of the triplet. We conduct experiments on link prediction with two benchmark datasets (WN18RR and FB15k-237). The results show that our method can effectively model the polysemy and uncertainty of entities and relations in KG.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, M., Zhang, Y., Qiu, M., et al.: SPHA: smart personal health advisor based on deep analytics. IEEE Commun. Mag. 56(3), 164–169 (2018)
Tao, L., Golikov, S., Gai, K., et al.: A reusable software component for integrated syntax and semantic validation for services computing. In: 2015 IEEE Symposium on Service-Oriented System Engineering (SOSE), pp. 127–132. IEEE (2015)
Gai, K., Qiu, M.: Reinforcement learning-based content-centric services in mobile sensing. IEEE Netw. 32(4), 34–39 (2018)
He, S., Liu, K., Ji, G., Zhao, J.: Learning to represent knowledge graphs with Gaussian embedding. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 623–632. ACM (2015)
Xiao, H., Huang, M., Zhu, X.: TransG: a generative model for knowledge graph embedding. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, pp. 2316–2325. ACL (2016)
Athiwaratkun, B., Wilson, A.: Multimodal word distributions. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, pp. 1645–1656. ACL(2017)
Bordes, A., Usunier, N., Garcia-Duran, A., et al.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)
Wang, Z., Zhang, J., Feng, J., et al.: Knowledge graph embedding by translating on hyperplanes. In: 28th AAAI Conference on Artificial Intelligence (2014)
Lin, Y., Liu, Z., Sun, M., et al.: Learning entity and relation embeddings for knowledge graph completion. In: 29th AAAI Conference on Artificial Intelligence (2015)
Ji, G., He, S., Xu, L., et al.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 687–696 (2015)
Fan, M., Zhou, Q., Chang, E., et al.: Transition-based knowledge graph embedding with relational mapping properties. In: Proceedings of the 28th Pacific Asia Conference on Language, Information and Computation, pp. 328–337 (2014)
Nickel, M., Tresp, V., Kriegel, H P.: A three-Way model for collective learning on multi-relational data. In: Proceedings of the 54th International Conference on Machine Learning, pp. 809–816 (2011)
Yang, B., Yih, W T., He, X., et al.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 3rd International Conference on Learning Representations (2015)
Trouillon, T., Welbl, J., Riedel, S., et al.: Complex embeddings for simple link prediction. In: Proceedings of the 33rd International Conference on Machine Learning, pp. 2071–2080 (2016)
Nickel, M., Rosasco, L., Poggio, T.: Holographic embeddings of knowledge graphs. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence, pp. 1955–1961 (2016)
Dettmers, T., Minervini, P., Stenetorp, P., et al.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (2018)
Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., et al.: A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers) (2018)
Che, F., Zhang, D., Tao, J., et al.: ParamE: regarding neural network parameters as relation embeddings for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2774–2781 (2020)
Vashishth, S., Sanyal, S., Nitin, V., et al.: InteractE: improving convolution-based knowledge graph embeddings by increasing feature interactions. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3009–3016 (2020)
Vilnis, L., McCallum, A.: Word representations via gaussian embedding. In: Proceedings of the International Conference on Learning Representations (2015)
Acknowledgements
This work is supported by the Youth Innovation Promotion Association, Chinese Academy of Sciences (No. 2017213). In addition, we would like to thank the anonymous reviewers for their valuable comments and suggestions that help improving the quality of this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Feng, W., Zha, D., Guo, X., Dong, Y., He, Y. (2021). Representing Knowledge Graphs with Gaussian Mixture Embedding. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management. KSEM 2021. Lecture Notes in Computer Science(), vol 12815. Springer, Cham. https://doi.org/10.1007/978-3-030-82136-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-82136-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82135-7
Online ISBN: 978-3-030-82136-4
eBook Packages: Computer ScienceComputer Science (R0)