Advertisement

Exploring the Generalization of Knowledge Graph Embedding

  • Liang Zhang
  • Huan GaoEmail author
  • Xianda Zheng
  • Guilin Qi
  • Jiming Liu
Conference paper
  • 8 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12032)

Abstract

Knowledge graph embedding aims to represent structured entities and relations as continuous and dense low-dimensional vectors. With more and more embedding models being proposed, it has been widely used in many tasks such as semantic search, knowledge graph completion and intelligent question and answer. Most knowledge graph embedding models focus on how to get information about different entities and relations. However, the generalization of knowledge graph embedding or the link prediction ability is not well-studied empirically and theoretically. The study of generalization ability is conducive to further improving the performance of the model. In this paper, we propose two measures to quantify the generalization ability of knowledge graph embedding and use them to analyze the performance of translation-based models. Extensive experimental results show that our measures can well evaluate the generalization ability of a knowledge graph embedding model.

Notes

Acknowledgment

Research presented in this paper was partially supported by the National Key Research and Development Program of China under grants (2018YFC0830200, 2017YFB1002801), the Natural Science Foundation of China grants (U1736204), the Judicial Big Data Research Centre, School of Law at Southeast University.

References

  1. 1.
    Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080 (2016)Google Scholar
  2. 2.
    Szumlanski, S., Gomez, F.: Automatically acquiring a semantic network of related concepts. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pp. 19–28. ACM (2010)Google Scholar
  3. 3.
    Xiao, H., Huang, M., Meng, L., Zhu, X.: SSP: semantic space projection for knowledge graph embedding with text descriptions. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)Google Scholar
  4. 4.
    Shi, B., Weninger, T.: ProjE: embedding projection for knowledge graph completion. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)Google Scholar
  5. 5.
    Guan, S., Jin, X., Wang, Y., Cheng, X.: Shared embedding based neural networks for knowledge graph completion. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 247–256. ACM (2018)Google Scholar
  6. 6.
    Neyshabur, B., Tomioka, R., Srebro, N.: In search of the real inductive bias: on the role of implicit regularization in deep learning. arXiv preprint arXiv:1412.6614 (2014)
  7. 7.
    Keskar, N.S., Mudigere, D., Nocedal, J., Smelyanskiy, M., Tang, P.T.P.: On large-batch training for deep learning: generalization gap and sharp minima. arXiv preprint arXiv:1609.04836 (2016)
  8. 8.
    Neyshabur, B., Salakhutdinov, R.R., Srebro, N.: Path-SGD: path-normalized optimization in deep neural networks. In: Advances in Neural Information Processing Systems, pp. 2422–2430 (2015)Google Scholar
  9. 9.
    Hoffer, E., Hubara, I., Soudry, D.: Train longer, generalize better: closing the generalization gap in large batch training of neural networks. In: Advances in Neural Information Processing Systems, pp. 1731–1741 (2017)Google Scholar
  10. 10.
    Neyshabur, B., Tomioka, R., Srebro, N.: Norm-based capacity control in neural networks. In: Conference on Learning Theory, pp. 1376–1401 (2015)Google Scholar
  11. 11.
    Sharma, A., Talukdar, P., et al.: Towards understanding the geometry of knowledge graph embeddings. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 122–131 (2018)Google Scholar
  12. 12.
    Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)Google Scholar
  13. 13.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (2014)Google Scholar
  14. 14.
    Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)Google Scholar
  15. 15.
    Xiao, H., Huang, M., Hao, Y., Zhu, X.: TransA: an adaptive approach for knowledge graph embedding. arXiv preprint arXiv:1509.05490 (2015)
  16. 16.
    Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 687–696 (2015)Google Scholar
  17. 17.
    Neyshabur, B., Bhojanapalli, S., McAllester, D., Srebro, N.: Exploring generalization in deep learning. In: Advances in Neural Information Processing Systems, pp. 5947–5956 (2017)Google Scholar
  18. 18.
    Kawaguchi, K., Kaelbling, L.P., Bengio, Y.: Generalization in deep learning. arXiv preprint arXiv:1710.05468 (2017)
  19. 19.
    Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)
  20. 20.
    Dragomir, S.S.: A survey on Cauchy-Bunyakovsky-Schwarz type discrete inequalities. J. Inequal. Pure Appl. Math. 4(3), 1–142 (2003)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250. ACM (2008)Google Scholar
  22. 22.
    Han, X., et al.: OpenKE: an open toolkit for knowledge embedding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 139–144 (2018)Google Scholar
  23. 23.
    Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds) Proceedings of COMPSTAT 2010. Physica-Verlag HD, Heidelberg (2010).  https://doi.org/10.1007/978-3-7908-2604-3_16CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Liang Zhang
    • 1
  • Huan Gao
    • 1
    Email author
  • Xianda Zheng
    • 2
  • Guilin Qi
    • 1
    • 2
  • Jiming Liu
    • 3
  1. 1.School of Computer Science and EngineeringSoutheast UniversityNanjingChina
  2. 2.School of Cyber Science and EngineeringSoutheast UniversityNanjingChina
  3. 3.Itibia TechnologiesSuzhouChina

Personalised recommendations