Abstract
The embedding representation technology provides convenience for machine learning on knowledge graphs (KG), which encodes entities and relations into continuous vector spaces and then constructs \(\langle entity,relation,entity \rangle \) triples. However, KG embedding models are sensitive to infrequent and uncertain objects. Furthermore, there is a contradiction between learning ability and learning cost. To this end, we propose circular embeddings (CirE) to learn representations of entire KG, which can accurately model various objects, save storage space, speed up calculation, and is easy to train and scalable to very large datasets. We have the following contributions: (1) We improve the accuracy of learning various objects by combining holographic projection and dynamic learning. (2) We reduce parameters and storage by adopting the circulant matrix as the projection matrix from the entity space to the relation space. (3) We reduce training time through adaptive parameters update algorithm which dynamically changes learning time for various objects. (4) We speed up the computation and enhance scalability by fast Fourier transform (FFT). Extensive experiments show that CirE outperforms state-of-the-art baselines in link prediction and entity classification, justifying the efficiency and the scalability of CirE.
This research was partially supported by the National Key Research and Development Program of China (No. 2016YFB1000603, 2016YFB1000602); the grants from the Natural Science Foundation of China (No. 61532010, 61379050, 91646203, 61532016); Specialized Research Fund for the Doctoral Program of Higher Education (No. 20130004130001), and the Fundamental Research Funds for the Central Universities, the Research Funds of Renmin University (No. 11XNL010).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Hit@10:proportion of correct entities in top-10 ranked entities.
References
Bordes, A., Usunier, N., Garcia-Duran, A, et al.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795. MIT Press Massachusetts (2013)
Ji, G., Liu, K., He, S., et al.: Knowledge graph completion with adaptive sparse transfer matrix. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 985–991. AAAI Press, MenloPark (2016)
Lin, Y., Liu, Z., Sun, M., et al.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 2181–2187. AAAI Press, MenloPark (2015)
Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on Machine Learning, pp. 809–816. ACM Press, New York (2011)
Nickel, M., Tresp, V., Kriegel, H.: Factorizing YAGO: scalable machine learning for linked data. In: Proceedings of the 21st World Wide Web Conference, pp. 271–280. ACM, New York (2012)
Nickel, M., Rosasco, L., Poggio, T.: Holographic embeddings of knowledge graphs. In: Proceedings of the Thirteenth AAAI Conference on Artificial Intelligence, pp. 1955–1961. AAAI Press, MenloPark (2016)
Wang, Z., Zhang, J., Feng, J., et al.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pp. 1112–1119. AAAI Press, MenloPark (2014)
Ji, G., He, S., Xu, L., et al.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pp. 687–696. MIT Press, Massachusetts (2015)
Sutskever, I., Tenenbaum, J.B., Salakhutdinov, R.: Modelling relational data using Bayesian clustered tensor factorization. In: Proceedings of Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems, pp. 1821–1828. MIT Press, Massachusetts (2009)
Jenatton, R., Roux, N.L., Bordes, A., et al.: A latent factor model for highly multi-relational data. In: Proceedings of Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems, pp. 3167–3175. MIT Press, Massachusetts (2012)
Yang, B., Yih, W., He, X., Entities, E., et al.: Relations for learning, inference in knowledge bases. In: Proceedings of ICLR (2015). Engelmore, R., Morgan, A. (eds.): Blackboard Systems. Addison-Wesley, Reading (1986)
Bordes, A., Weston, J., Collobert, R., et al.: Learning structured embeddings of knowledge bases. In: Proceedings of the the Twenty-Fifth AAAI Conference on Artificial Intelligence, pp. 301–306. AAAI Press, MenloPark (2011)
Socher, R., Chen, D., Manning, C.D., et al.: Reasoning with neural tensor networks for knowledge base completion, In: Proceedings of Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems, pp. 926–934. MIT Press, Massachusetts (2013)
Erbas, C., Tanik, M.M., Nair, V.S.S.: A circulant matrix based approach to storage schemes for parallel memory systems. In: Proceedings of the Fifth IEEE Symposium on Parallel and Distributed Processing, pp. 92–99. IEEE Press, Piscataway (1993)
Plate, T.A.: Holographic reduced representations. IEEE Trans. Neural Netw. 6(3), 623–641 (1995)
Zhang, J., Fu, N., Peng, X.: Compressive circulant matrix based analog to information conversion. IEEE Sig. Process. Lett. 21(4), 428–431 (2014)
Gentner, D.: Structure-mapping: a theoretical framework for analogy. Cogn. Sci. 7(2), 155–170 (1983)
Gabor, D.: Associative holographic memories. IBM J. Res. Dev. 13(2), 156–159 (1969)
Schnemann, P.H.: Some algebraic relations between involutions, convolutions, and correlations, with applications to holographic memories. Biol. Cybern. 56(5–6), 367–374 (1987)
Angel, E.S.: Fast Fourier transform and convolution algorithm. Proc. IEEE 70(5), 527–527 (1982)
Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning, stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012)
Bordes, A., Glorot, X., Weston, J., et al.: A semantic matching energy function for learning with multi-relational data. Mach. Learn. 94(2), 233–259 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Du, Z., Hao, Z., Meng, X., Wang, Q. (2017). CirE: Circular Embeddings of Knowledge Graphs. In: Candan, S., Chen, L., Pedersen, T., Chang, L., Hua, W. (eds) Database Systems for Advanced Applications. DASFAA 2017. Lecture Notes in Computer Science(), vol 10177. Springer, Cham. https://doi.org/10.1007/978-3-319-55753-3_10
Download citation
DOI: https://doi.org/10.1007/978-3-319-55753-3_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-55752-6
Online ISBN: 978-3-319-55753-3
eBook Packages: Computer ScienceComputer Science (R0)