Skip to main content
Book cover

International Semantic Web Conference

ISWC 2021: The Semantic Web – ISWC 2021 pp 410–426Cite as

Improving Knowledge Graph Embeddings with Ontological Reasoning

Part of the Lecture Notes in Computer Science book series (LNISA,volume 12922)

Abstract

Knowledge graph (KG) embedding models have emerged as powerful means for KG completion. To learn the representation of KGs, entities and relations are projected in a low-dimensional vector space so that not only existing triples in the KG are preserved but also new triples can be predicted. Embedding models might learn a good representation of the input KG, but due to the nature of machine learning approaches, they often lose the semantics of entities and relations, which might lead to nonsensical predictions. To address this issue we propose to improve the accuracy of embeddings using ontological reasoning. More specifically, we present a novel iterative approach ReasonKGE that identifies dynamically via symbolic reasoning inconsistent predictions produced by a given embedding model and feeds them as negative samples for retraining this model. In order to address the scalability problem that arises when integrating ontological reasoning into the training process, we propose an advanced technique to generalize the inconsistent predictions to other semantically similar negative samples during retraining. Experimental results demonstrate the improvements in accuracy of facts produced by our method compared to the state-of-the-art.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-88361-4_24
  • Chapter length: 17 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-88361-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.

Notes

  1. 1.

    https://github.com/nitishajain/ReasonKGE.

  2. 2.

    https://www.w3.org/TR/owl2-overview/.

  3. 3.

    Available at https://github.com/nitishajain/ReasonKGE.

  4. 4.

    For each triple the subject (resp. object) is randomly perturbed to obtain m samples [9].

References

  1. Ahrabian, K., Feizi, A., Salehi, Y., Hamilton, W.L., Bose, A.J.: Structure-aware negative sampling in knowledge graphs. EMNLP 2020, 6093–6101 (2020)

    Google Scholar 

  2. Alam, M.M., Jabeen, H., Ali, M., Mohiuddin, K., Lehmann, J.: Affinity dependent negative sampling for knowledge graph embeddings. In: (DL4KG2020) - (ESWC 2020) (2020)

    Google Scholar 

  3. Artale, A., Calvanese, D., Kontchakov, R., Zakharyaschev, M.: The DL-Lite family and relations. CoRR abs/1401.3487 (2014)

    Google Scholar 

  4. Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., Ives, Z.G.: Dbpedia: a nucleus for a web of open data. In: ISWC, pp. 722–735 (2007)

    Google Scholar 

  5. Baader, F., Horrocks, I., Sattler, U.: Description logics. In: Hb on Ontology, pp. 21–43 (2009)

    Google Scholar 

  6. Bianchi, F., Rossiello, G., Costabello, L., Palmonari, M., Minervini, P.: Knowledge graph embeddings and explainable AI. In: Tiddi, I., Lécué, F., Hitzler, P. (eds.) KGs for XAI: Foundations, Applications and Challenges, vol. 47, pp. 49–72. IOS Press (2020)

    Google Scholar 

  7. Bienvenu, M.: A short survey on inconsistency handling in ontology-mediated query answering. Künstliche Intell. 34(4), 443–451 (2020)

    CrossRef  Google Scholar 

  8. Bischof, S., Krötzsch, M., Polleres, A., Rudolph, S.: Schema-agnostic query rewriting in SPARQL 1.1. In: ISWC, pp. 584–600 (2014)

    Google Scholar 

  9. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NeurIPS, pp. 2787–2795 (2013)

    Google Scholar 

  10. Cai, L., Wang, W.Y.: KBGAN: adversarial learning for knowledge graph embeddings. NAACL-HLT 2018, 1470–1480 (2018)

    Google Scholar 

  11. d’Amato, C., Quatraro, N.F., Fanizzi, N.: Injecting background knowledge into embedding models for predictive tasks on knowledge graphs. In: ESWC, to appear (2021)

    Google Scholar 

  12. Dash, S., Gliozzo, A.: Distributional negative sampling for knowledge base completion. CoRR abs/1908.06178 (2019)

    Google Scholar 

  13. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: AAAI, pp. 1811–1818 (2018)

    Google Scholar 

  14. Garg, D., Ikbal, S., Srivastava, S.K., Vishwakarma, H., Karanam, H.P., Subramaniam, L.V.: Quantum embedding of knowledge for reasoning. In: NeurIPS, pp. 5595–5605 (2019)

    Google Scholar 

  15. Glimm, B., Horrocks, I., Motik, B., Stoilos, G., Wang, Z.: Hermit: an OWL 2 reasoner. J. Autom. Reasoning 53(3), 245–269 (2014)

    CrossRef  Google Scholar 

  16. Glimm, B., Kazakov, Y., Liebig, T., Tran, T.K., Vialard, V.: ISWC, pp. 180–195 (2014)

    Google Scholar 

  17. Glimm, B., Kazakov, Y., Tran, T.: Ontology materialization by abstraction refinement in horn SHOIF. In: AAAI, pp. 1114–1120 (2017)

    Google Scholar 

  18. Guo, Y., Pan, Z., Heflin, J.: LUBM: a benchmark for OWL knowledge base systems. J. Web Semant. 3(2–3), 158–182 (2005)

    CrossRef  Google Scholar 

  19. Hao, J., Chen, M., Yu, W., Sun, Y., Wang, W.: Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts. In: KDD, pp. 1709–1719 (2019)

    Google Scholar 

  20. Horridge, M., Parsia, B., Sattler, U.: Explaining inconsistencies in owl ontologies. In: Scalable Uncertainty Management, pp. 124–137 (2009)

    Google Scholar 

  21. Kotnis, B., Nastase, V.: Analysis of the impact of negative sampling on link prediction in knowledge graphs. CoRR abs/1708.06816 (2017). http://arxiv.org/abs/1708.06816

  22. Krompaß, D., Baier, S., Tresp, V.: Type-constrained representation learning in knowledge graphs. In: ISWC, pp. 640–655 (2015)

    Google Scholar 

  23. Lembo, D., Lenzerini, M., Rosati, R., Ruzzi, M., Savo, D.F.: Inconsistency-tolerant query answering in ontology-based data access. J. Web Semant. 33, 3–29 (2015)

    CrossRef  Google Scholar 

  24. Liu, Y., Li, H., Garcia-Duran, A., Niepert, M., Onoro-Rubio, D., Rosenblum, D.S.: MMKG: multi-modal knowledge graphs. In: ESWC, pp. 459–474 (2019)

    Google Scholar 

  25. Minervini, P., Demeester, T., Rocktäschel, T., Riedel, S.: Adversarial sets for regularising neural link predictors. In: UAI (2017)

    Google Scholar 

  26. Motik, B., Patel-Schneider, P.F., Grau, B.C.: OWL 2 web ontology language direct semantics (Second Edition). Tech. rep. (2012). https://www.w3.org/TR/owl-direct-semantics/

  27. Nickel, M., Murphy, K., Tresp, V., Gabrilovich, E.: A review of relational machine learning for knowledge graphs. Proc. IEEE 104(1), 11–33 (2016)

    CrossRef  Google Scholar 

  28. Paulheim, H., Gangemi, A.: Serving DBpedia with DOLCE – more than just adding a cherry on top. In: Arenas, M., et al. (eds.) ISWC 2015. LNCS, vol. 9366, pp. 180–196. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-25007-6_11

    CrossRef  Google Scholar 

  29. Ruffinelli, D., Broscheit, S., Gemulla, R.: You CAN teach an old dog new tricks! on training knowledge graph embeddings. In: ICLR (2020)

    Google Scholar 

  30. Socher, R., Chen, D., Manning, C.D., Ng, A.Y.: Reasoning with neural tensor networks for knowledge base completion. In: NIPS. pp. 926–934 (2013)

    Google Scholar 

  31. Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: WWW (2007)

    Google Scholar 

  32. Tran, T., Gad-Elrab, M.H., Stepanova, D., Kharlamov, E., Strötgen, J.: Fast computation of explanations for inconsistency in large-scale KGS. In: WWW, vol. 2020, pp. 2613–2619 (2020)

    Google Scholar 

  33. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: ICML, pp. 2071–2080 (2016)

    Google Scholar 

  34. Vrandecic, D., Krötzsch, M.: Wikidata: a free collaborative knowledgebase. Commun. ACM 57(10), 78–85 (2014)

    CrossRef  Google Scholar 

  35. Wang, P., Li, S., Pan, R.: Incorporating GAN for negative sampling in knowledge representation learning. In: AAAI, pp. 2005–2012 (2018)

    Google Scholar 

  36. Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017)

    CrossRef  Google Scholar 

  37. Wiharja, K., Pan, J.Z., Kollingbaum, M.J., Deng, Y.: Schema aware iterative knowledge graph completion. J. Web Semant. 65, 100616 (2020)

    CrossRef  Google Scholar 

  38. Zhang, J., Chen, B., Zhang, L., Ke, X., Ding, H.: Neural-symbolic reasoning on knowledge graphs. CoRR abs/2010.05446 (2020)

    Google Scholar 

  39. Zhang, Y., Yao, Q., Chen, L.: Efficient, simple and automated negative sampling for knowledge graph embedding. CoRR abs/2010.14227 (2020), https://arxiv.org/abs/2010.14227

  40. Zhang, Y., Yao, Q., Shao, Y., Chen, L.: Nscaching: simple and efficient negative sampling for knowledge graph embedding. In: ICDE, pp. 614–625 (2019)

    Google Scholar 

  41. Ziegler, K., et al.: Injecting semantic background knowledge into neural networks using graph embeddings. In: 26th IEEE, WETICE, pp. 200–205 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bosch Center for Artificial Intelligence, Renningen, Germany

    Nitisha Jain, Trung-Kien Tran, Mohamed H. Gad-Elrab & Daria Stepanova

  2. Hasso Plattner Institute, University of Potsdam, Potsdam, Germany

    Nitisha Jain

Authors
  1. Nitisha Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Trung-Kien Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed H. Gad-Elrab
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daria Stepanova
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nitisha Jain .

Editor information

Editors and Affiliations

  1. University of Würzburg, Würzburg, Germany

    Prof. Dr. Andreas Hotho

  2. Linköping University, Linköping, Sweden

    Assoc. Prof. Eva Blomqvist

  3. University of Düsseldorf, Düsseldorf, Germany

    Stefan Dietze

  4. IBM Research - Thomas J. Watson Research, Hawthorne, CA, USA

    Achille Fokoue

  5. University of Texas, Austin, TX, USA

    Ying Ding

  6. Imperial College, London, UK

    Payam Barnaghi

  7. Australian National University, Canberra, ACT, Australia

    Armin Haller

  8. Fondazione Bruno Kessler, Povo, Trento, Italy

    Dr. Mauro Dragoni

  9. The Open University Walton Hall, Milton Keynes, UK

    Harith Alani

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Jain, N., Tran, TK., Gad-Elrab, M.H., Stepanova, D. (2021). Improving Knowledge Graph Embeddings with Ontological Reasoning. In: , et al. The Semantic Web – ISWC 2021. ISWC 2021. Lecture Notes in Computer Science(), vol 12922. Springer, Cham. https://doi.org/10.1007/978-3-030-88361-4_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-88361-4_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-88360-7

  • Online ISBN: 978-3-030-88361-4

  • eBook Packages: Computer ScienceComputer Science (R0)