Skip to main content

Do Embeddings Actually Capture Knowledge Graph Semantics?

Part of the Lecture Notes in Computer Science book series (LNISA,volume 12731)

Abstract

Knowledge graph embeddings that generate vector space representations of knowledge graph triples, have gained considerable popularity in past years. Several embedding models have been proposed that achieve state-of-the-art performance for the task of triple completion in knowledge graphs. Relying on the presumed semantic capabilities of the learned embeddings, they have been leveraged for various other tasks such as entity typing, rule mining and conceptual clustering. However, a critical analysis of the utility as well as limitations of these embeddings for semantic representation of the underlying entities and relations has not been performed by previous work.

In this paper, we performed a systematic evaluation of popular knowledge graph embedding models to obtain a better understanding of their semantic capabilities as compared to a non-embedding based approach. Our analysis brings attention to the fact that semantic representation in the knowledge graph embeddings is not universal, but restricted to a small subset of the entities based on dataset characteristics. We provide further insights into the reasons for this behaviour. The results of our experiments indicate that careful analysis of benefits of the embeddings needs to be performed when employing them for semantic tasks.

Keywords

  • Knowledge graph embeddings
  • Semantic representation
  • Entity similarity

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-77385-4_9
  • Chapter length: 17 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-77385-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   139.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

Notes

  1. 1.

    https://github.com/nitishajain/KGESemanticAnalysis.

  2. 2.

    The training parameters and performance scores are available on github link.

  3. 3.

    https://github.com/IBCNServices/pyRDF2Vec.

  4. 4.

    ARI and V-measure show similar trend, full results are available on github link.

References

  1. Akrami, F., Saeef, M.S., Zhang, Q., Hu, W., Li, C.: Realistic re-evaluation of knowledge graph completion methods: an experimental study, pp. 1995–2010 (2020)

    Google Scholar 

  2. Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.: Dbpedia: a nucleus for a web of open data. In: International Semantic Web Conference, pp. 722–735 (2007)

    Google Scholar 

  3. Bhowmik, R., de Melo, G.: Explainable link prediction for emerging entities in knowledge graphs. In: Pan, J.Z., Tamma, V., d’Amato, C., Janowicz, K., Fu, B., Polleres, A., Seneviratne, O., Kagal, L. (eds.) ISWC 2020. LNCS, vol. 12506, pp. 39–55. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-62419-4_3

    CrossRef  Google Scholar 

  4. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250 (2008)

    Google Scholar 

  5. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)

    Google Scholar 

  6. Broscheit, S., Ruffinelli, D., Kochsiek, A., Betz, P., Gemulla, R.: LibKGE - A knowledge graph embedding library for reproducible research. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 165–174 (2020)

    Google Scholar 

  7. Chen, W., Zhu, H., Han, X., Liu, Z., Sun, M.: Quantifying similarity between relations with fact distribution. In: Proceedings of the 57th Conference of the Association for Computational Linguistics pp. 2882–2894 (2019)

    Google Scholar 

  8. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, vol. 32, pp. 1811–1818 (2018)

    Google Scholar 

  9. Gad-Elrab, M.H., Ho, V.T., Levinkov, E., Tran, T., Stepanova, D.: Towards utilizing knowledge graph embedding models for conceptual clustering. In: Proceedings of the ISWC 2020 Demos and Industry Tracks, vol. 2721, pp. 281–286 (2020)

    Google Scholar 

  10. Gad-Elrab, M.H., Stepanova, D., Tran, T.K., Adel, H., Weikum, G.: ExCut: explainable embedding-based clustering over knowledge graphs. In: International Semantic Web Conference, pp. 218–237 (2020)

    Google Scholar 

  11. Hao, J., Chen, M., Yu, W., Sun, Y., Wang, W.: Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1709–1719 (2019)

    Google Scholar 

  12. Ho, V.T., Stepanova, D., Gad-Elrab, M.H., Kharlamov, E., Weikum, G.: Rule learning from knowledge graphs guided by embedding models. In: International Semantic Web Conference, pp. 72–90 (2018)

    Google Scholar 

  13. Hussein, R., Yang, D., Cudré-Mauroux, P.: Are meta-paths necessary? revisiting heterogeneous graph embeddings. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 437–446 (2018)

    Google Scholar 

  14. Jain, N., Krestel, R.: Learning fine-grained semantics for multi-relational data. In: International Semantic Web Conference, 2020 Posters and Demos (2020)

    Google Scholar 

  15. Jain, P., Kumar, P., Chakrabarti, S., et al.: Type-sensitive knowledge base inference without explicit type supervision. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 75–80 (2018)

    Google Scholar 

  16. Kalo, J.C., Ehler, P., Balke, W.T.: Knowledge graph consolidation by unifying synonymous relationships. In: International Semantic Web Conference, pp. 276–292 (2019)

    Google Scholar 

  17. Mahdisoltani, F., Biega, J., Suchanek, F.: YAGO3: a knowledge base from multilingual Wikipedias. In: 7th Biennial Conference on Innovative Data Systems Research (2014)

    Google Scholar 

  18. Miller, G.A.: WordNet: a lexical database for english. Commun. ACM 38(11), 39–41 (1995)

    CrossRef  Google Scholar 

  19. Moon, C., Jones, P., Samatova, N.F.: Learning entity type embeddings for knowledge graph completion. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 2215–2218 (2017)

    Google Scholar 

  20. Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on Machine Learning, ICML 2011, pp. 809–816 (2011)

    Google Scholar 

  21. Nickel, M., Tresp, V., Kriegel, H.P.: Factorizing yago: scalable machine learning for linked data. In: Proceedings of the 21st International Conferen on World Wide Web, pp. 271–280 (2012)

    Google Scholar 

  22. Paulheim, H., Bizer, C.: Type inference on noisy RDF data. In: International Semantic Web Conference, pp. 510–525 (2013)

    Google Scholar 

  23. Ristoski, P., Paulheim, H.: Rdf2vec: Rdf graph embeddings for data mining. In: International Semantic Web Conference, pp. 498–514 (2016)

    Google Scholar 

  24. Rossi, A., Matinata, A.: Knowledge graph embeddings: are relation-learning models learning relations? In: EDBT/ICDT Workshops (2020)

    Google Scholar 

  25. Ruffinelli, D., Broscheit, S., Gemulla, R.: You \(\{\)can\(\}\) teach an old dog new tricks! on training knowledge graph embeddings. In: International Conference on Learning Representations (2020)

    Google Scholar 

  26. Sun, Z., Zhang, Q., Hu, W., Wang, C., Chen, M., Akrami, F., Li, C.: A benchmarking study of embedding-based entity alignment for knowledge graphs. Proc. VLDB Endow. 13(12), 2326–2340 (2020)

    CrossRef  Google Scholar 

  27. Toutanova, K., Chen, D.: Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pp. 57–66 (2015)

    Google Scholar 

  28. Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of the 33rd International Conference on Machine Learning - Volume 48, ICML 2016, pp. 2071–2080 (2016)

    Google Scholar 

  29. Wang, C., Pan, S., Hu, R., Long, G., Jiang, J., Zhang, C.: Attributed graph clustering: a deep attentional embedding approach (2019)

    Google Scholar 

  30. Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017)

    CrossRef  Google Scholar 

  31. Yang, B., Yih, S.W.t., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the International Conference on Learning Representations (ICLR) 2015, May 2015

    Google Scholar 

  32. Zhang, W., et al.: Iteratively learning embeddings and rules for knowledge graph reasoning. In: Proceedings of the 2019 World Wide Web Conference, WWW 2019, pp. 2366–2377 (2019)

    Google Scholar 

  33. Zhao, Y., Xie, R., Liu, K., Xiaojie, W., et al.: Connecting embeddings for knowledge graph entity typing. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6419–6428 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nitisha Jain .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Jain, N., Kalo, JC., Balke, WT., Krestel, R. (2021). Do Embeddings Actually Capture Knowledge Graph Semantics?. In: , et al. The Semantic Web. ESWC 2021. Lecture Notes in Computer Science(), vol 12731. Springer, Cham. https://doi.org/10.1007/978-3-030-77385-4_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77385-4_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77384-7

  • Online ISBN: 978-3-030-77385-4

  • eBook Packages: Computer ScienceComputer Science (R0)