Skip to main content
SpringerLink
Log in

Knowledge Graph Completion Using Structural and Textual Embeddings

  • Conference paper
  • First Online:
Artificial Intelligence Applications and Innovations (AIAI 2024)

Part of the book series: IFIP Advances in Information and Communication Technology ((IFIPAICT,volume 713))

  • 249 Accesses

Abstract

Knowledge Graphs (KGs) are widely employed in artificial intelligence applications, such as question-answering and recommendation systems. However, KGs are frequently found to be incomplete. While much of the existing literature focuses on predicting missing nodes for given incomplete KG triples, there remains an opportunity to complete KGs by exploring relations between existing nodes, a task known as relation prediction. In this study, we propose a relations prediction model that harnesses both textual and structural information within KGs. Our approach integrates walks-based embeddings with language model embeddings to effectively represent nodes. We demonstrate that our model achieves competitive results in the relation prediction task when evaluated on a widely used dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 299.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/sa5r/KGRP.

  2. 2.

    https://developers.google.com/freebase/.

References

  1. Alqaaidi, S.K., Bozorgi, E., Kochut, K.J.: Multiple relations classification using imbalanced predictions adaptation. arXiv preprint arXiv:2309.13718 (2023)

  2. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250 (2008)

    Google Scholar 

  3. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Proc. Syst. 26 (2013)

    Google Scholar 

  4. Chen, Z., Wang, Y., Zhao, B., Cheng, J., Zhao, X., Duan, Z.: Knowledge graph completion: a review. IEEE Access 8, 192435–192456 (2020)

    Article  Google Scholar 

  5. Cui, Z., Kapanipathi, P., Talamadupula, K., Gao, T., Ji, Q.: Type-augmented relation prediction in knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 7151–7159 (2021)

    Google Scholar 

  6. Daza, D., Cochez, M., Groth, P.: Inductive entity representations from text via link prediction. In: Proceedings of the Web Conference 2021, pp. 798–808 (2021)

    Google Scholar 

  7. Demir, C., Moussallem, D., Ngomo, A.C.N.: A shallow neural model for relation prediction. In: 2021 IEEE 15th International Conference on Semantic Computing (ICSC), pp. 179–182. IEEE (2021)

    Google Scholar 

  8. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  9. Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864 (2016)

    Google Scholar 

  10. Han, J., Moraga, C.: The influence of the sigmoid function parameters on the speed of backpropagation learning. In: Mira, J., Sandoval, F. (eds.) IWANN 1995. LNCS, vol. 930, pp. 195–201. Springer, Heidelberg (1995). https://doi.org/10.1007/3-540-59497-3_175

    Chapter  Google Scholar 

  11. He, J., Jia, L., Wang, L., Li, X., Xu, X.: MoCoSA: Momentum contrast for knowledge graph completion with structure-augmented pre-trained language models. arXiv preprint arXiv:2308.08204 (2023)

  12. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)

  13. Hu, W., et al.: Open graph benchmark: datasets for machine learning on graphs. Adv. Neural. Inf. Process. Syst. 33, 22118–22133 (2020)

    Google Scholar 

  14. Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S.: Modeling relation paths for representation learning of knowledge bases. arXiv preprint arXiv:1506.00379 (2015)

  15. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 29(1), (2015)

    Google Scholar 

  16. Liu, Y., et al.: RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

  17. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)

  18. Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  19. Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710 (2014)

    Google Scholar 

  20. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)

    Google Scholar 

  21. Sak, H., Senior, A., Beaufays, F.: Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv preprint arXiv:1402.1128 (2014)

  22. Shen, J., Wang, C., Gong, L., Song, D.: Joint language semantic and structure embedding for knowledge graph completion. arXiv preprint arXiv:2209.08721 (2022)

  23. Shen, T., Zhang, F., Cheng, J.: A comprehensive overview of knowledge graph completion. Knowl. Based Syst. 255, p. 109597 (2022)

    Google Scholar 

  24. Sheu, H.S., Li, S.: Context-aware graph embedding for session-based news recommendation. In: Proceedings of the 14th ACM Conference on Recommender Systems, pp. 657–662 (2020)

    Google Scholar 

  25. Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197 (2019)

  26. Tagawa, Y., et al.: Relation prediction for unseen-entities using entity-word graphs. In: Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13), pp. 11–16 (2019)

    Google Scholar 

  27. Touvron, H., et al.: Llama 2: Open foundation and fine-tuned chat models (2023). https://arxiv.org/abs/2307.09288 (2023)

  28. Vaswani, A., et al.: Attention is all you need. In: Advance Neural Information Processing System vol. 30 (2017)

    Google Scholar 

  29. Wang, B., Shen, T., Long, G., Zhou, T., Wang, Y., Chang, Y.: Structure-augmented text representation learning for efficient knowledge graph completion. In: Proceedings of the Web Conference 2021, pp. 1737–1748 (2021)

    Google Scholar 

  30. Wang, L., Zhao, W., Wei, Z., Liu, J.: Simkgc: simple contrastive knowledge graph completion with pre-trained language models. arXiv preprint arXiv:2203.02167 (2022)

  31. Wang, X., et al.: KEPLER: a unified model for knowledge embedding and pre-trained language representation. Trans. Assoc. Comput. Linguist. 9, 176–194 (2021)

    Article  Google Scholar 

  32. Xu, J., Chen, K., Qiu, X., Huang, X.: Knowledge graph representation with jointly structural and textual encoding. arXiv preprint arXiv:1611.08661 (2016)

  33. Yani, M., Krisnadhi, A.A.: Challenges, techniques, and trends of simple knowledge graph question answering: a survey. Information 12(7), 271 (2021)

    Article  Google Scholar 

  34. Yao, L., Mao, C., Luo, Y.: KG-BERT: BERT for knowledge graph completion. arXiv preprint arXiv:1909.03193 (2019)

  35. Youn, J., Tagkopoulos, I.: KGLM: Integrating knowledge graph structure in language models for link prediction. arXiv preprint arXiv:2211.02744 (2022)

  36. Zha, H., Chen, Z., Yan, X.: Inductive relation prediction by BERT. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 5923–5931 (2022)

    Google Scholar 

  37. Zhuang, Z., Liang, Z., Rao, Y., Xie, H., Wang, F.L.: Out-of-vocabulary word embedding learning based on reading comprehension mechanism. Nat. Lang. Process. J. 5, 100038 (2023)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, University of Georgia, Athens, Georgia

    Sakher Khalil Alqaaidi & Krzysztof J. Kochut

Authors
  1. Sakher Khalil Alqaaidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Krzysztof J. Kochut
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sakher Khalil Alqaaidi .

Editor information

Editors and Affiliations

  1. University of Piraeus, Piraeus, Greece

    Ilias Maglogiannis

  2. Democritus University of Thrace, Xanthi, Greece

    Lazaros Iliadis

  3. University of Abertay, Dundee, UK

    John Macintyre

  4. Ionian University, Corfu, Greece

    Markos Avlonitis

  5. Democritus University of Thrace, Xanthi, Greece

    Antonios Papaleonidas

Rights and permissions

Reprints and permissions

Copyright information

© 2024 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alqaaidi, S.K., Kochut, K.J. (2024). Knowledge Graph Completion Using Structural and Textual Embeddings. In: Maglogiannis, I., Iliadis, L., Macintyre, J., Avlonitis, M., Papaleonidas, A. (eds) Artificial Intelligence Applications and Innovations. AIAI 2024. IFIP Advances in Information and Communication Technology, vol 713. Springer, Cham. https://doi.org/10.1007/978-3-031-63219-8_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-63219-8_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-63218-1

  • Online ISBN: 978-3-031-63219-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation