Skip to main content
View expanded cover

International Semantic Web Conference

ISWC 2020: The Semantic Web – ISWC 2020 pp 39–55Cite as

Explainable Link Prediction for Emerging Entities in Knowledge Graphs

Part of the Lecture Notes in Computer Science book series (LNISA,volume 12506)

Abstract

Despite their large-scale coverage, cross-domain knowledge graphs invariably suffer from inherent incompleteness and sparsity. Link prediction can alleviate this by inferring a target entity, given a source entity and a query relation. Recent embedding-based approaches operate in an uninterpretable latent semantic vector space of entities and relations, while path-based approaches operate in the symbolic space, making the inference process explainable. However, these approaches typically consider static snapshots of the knowledge graphs, severely restricting their applicability for evolving knowledge graphs with newly emerging entities. To overcome this issue, we propose an inductive representation learning framework that is able to learn representations of previously unseen entities. Our method finds reasoning paths between source and target entities, thereby making the link prediction for unseen entities interpretable and providing support evidence for the inferred link.

Keywords

  • Explainable link prediction
  • Emerging entities
  • Inductive representation learning

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-62419-4_3
  • Chapter length: 17 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-62419-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   139.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.

Notes

  1. 1.

    From here onwards, we will use the terms node and entity, as well as edge and relation(ship) interchangeably.

  2. 2.

    https://github.com/pykeen/pykeen.

  3. 3.

    https://github.com/dmlc/dgl/tree/master/examples/pytorch/rgcn.

  4. 4.

    https://github.com/salesforce/MultiHopKG.

References

  1. Ba, J., Kiros, J.R., Hinton, G.E.: Layer normalization. ArXiv, vol. 1607, p. 06450 (2016)

    Google Scholar 

  2. Bhowmik, R., de Melo, G.: Be concise and precise: synthesizing open-domain entity descriptions from facts. In: Proceedings of The Web Conference 2019, pp. 116–126. ACM, New York (2019)

    Google Scholar 

  3. Bordes, A., Usunier, N., García-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Adv. Neural Inform. Process. Syst. 26, 2787–2795 (2013)

    Google Scholar 

  4. Das, R., et al.: Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. arXiv 1711.05851 (2017)

    Google Scholar 

  5. Dasgupta, S.S., Ray, S.N., Talukdar, P.: HyTE: hyperplane-based temporally aware knowledge graph embedding. In: Proceedings of EMNLP 2018 (2018)

    Google Scholar 

  6. van Erp, M., et al. (eds.): ISWC 2016. LNCS, vol. 10579. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68723-0

    CrossRef  Google Scholar 

  7. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI 2018), pp. 1811–1818. AAAI Press (2018)

    Google Scholar 

  8. Fu, Z., et al.: Fairness-aware explainable recommendation over knowledge graphs. In: Proceedings of the 43rd SIGIR 2020. ACM (2020)

    Google Scholar 

  9. Galárraga, L., Teflioudi, C., Hose, K., Suchanek, F.M.: Fast rule mining in ontological knowledge bases with AMIE++. VLDB J. 24(6), 707–730 (2015)

    CrossRef  Google Scholar 

  10. Gardner, M., Talukdar, P.P., Kisiel, B., Mitchell, T.M.: Improving learning and inference in a large knowledge-base using latent syntactic cues. In: Proceedings of EMNLP 2013, pp. 833–838. ACL (2013)

    Google Scholar 

  11. Gardner, M., Talukdar, P.P., Krishnamurthy, J., Mitchell, T.M.: Incorporating vector space similarity in random walk inference over knowledge bases. In: Proceedings of EMNLP 2014, pp. 397–406. ACL (2014)

    Google Scholar 

  12. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, vol. 9, pp. 249–256. PMLR, 13–15 May 2010

    Google Scholar 

  13. Guu, K., Miller, J., Liang, P.: Traversing knowledge graphs in vector space. In: Proceedings of EMNLP 2015, pp. 318–327. ACL (2015)

    Google Scholar 

  14. Hamaguchi, T., Oiwa, H., Shimbo, M., Matsumoto, Y.: Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. In: Proceedings of IJCAI, pp. 1802–1808. AAAI Press (2017)

    Google Scholar 

  15. Hamilton, W.L., Ying, R., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing (2017)

    Google Scholar 

  16. Ho, V.T., Stepanova, D., Gad-Elrab, M.H., Kharlamov, E., Weikum, G.: Rule learning from knowledge graphs guided by embedding models. In: Vrandečić, D., et al. (eds.) ISWC 2018. LNCS, vol. 11136, pp. 72–90. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00671-6_5

    CrossRef  Google Scholar 

  17. Hogan, A., et al.: Knowledge graphs. ArXiv, vol. 2003, p. 02320 (2020)

    Google Scholar 

  18. Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of ACL-IJCNLP 2015, pp. 687–696. ACL (2015)

    Google Scholar 

  19. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). http://arxiv.org/abs/1412.6980

  20. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (ICLR) (2017)

    Google Scholar 

  21. Koncel-Kedziorski, R., Bekal, D., Luan, Y., Lapata, M., Hajishirzi, H.: Text generation from knowledge graphs with graph transformers. In: Proceedings of NAACL 2019, pp. 2284–2293. ACL, June 2019

    Google Scholar 

  22. Lao, N., Mitchell, T.M., Cohen, W.W.: Random walk inference and learning in a large scale knowledge base. In: Proceedings of EMNLP 2011, pp. 529–539. ACL (2011)

    Google Scholar 

  23. Lin, X.V., Socher, R., Xiong, C.: Multi-hop knowledge graph reasoning with reward shaping. arXiv abs/1808.10568 (2018). http://arxiv.org/abs/1808.10568

  24. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI Conference on Artificial Intelligence (2015)

    Google Scholar 

  25. Meilicke, C., Chekol, M.W., Ruffinelli, D., Stuckenschmidt, H.: An introduction to AnyBURL. In: Benzmüller, C., Stuckenschmidt, H. (eds.) KI 2019. LNCS (LNAI), vol. 11793, pp. 244–248. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30179-8_20

    CrossRef  Google Scholar 

  26. Neelakantan, A., Roth, B., McCallum, A.: Compositional vector space models for knowledge base completion. In: Proceedings of ACL 2015. ACL (2015)

    Google Scholar 

  27. Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., Phung, D.: A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of NAACL, vol. 2018, pp. 327–333 (2018)

    Google Scholar 

  28. Nguyen, D.Q.: An overview of embedding models of entities and relationships for knowledge base completion. arXiv 1703.08098 (2017)

    Google Scholar 

  29. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling Relational Data with Graph Convolutional Networks. In: Gangemi, A., Navigli, R., Vidal, M.-E., Hitzler, P., Troncy, R., Hollink, L., Tordai, A., Alam, M. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38

    CrossRef  Google Scholar 

  30. Shang, C., Tang, Y., Huang, J., Bi, J., He, X., Zhou, B.: End-to-end structure-aware convolutional networks for knowledge base completion. In: Proceedings of AAAI (2019)

    Google Scholar 

  31. Shen, Y., Chen, J., Huang, P.S., Guo, Y., Gao, J.: M-Walk: Learning to walk over graphs using Monte Carlo tree search. In: Advances in Neural Information Processing Systems 31, pp. 6786–6797. Curran Associates, Inc. (2018)

    Google Scholar 

  32. Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: RotatE: knowledge graph embedding by relational rotation in complex space. In: International Conference on Learning Representations (2019)

    Google Scholar 

  33. Toutanova, K., Lin, V., Yih, W., Poon, H., Quirk, C.: Compositional learning of embeddings for relation paths in knowledge base and text. In: Proceedings of ACL 2016. ACL (2016). http://aclweb.org/anthology/P/P16/P16-1136.pdf

  34. Trivedi, R., Farajtabar, M., Biswal, P., Zha, H.: DyRep: learning representations over dynamic graphs. In: ICLR (2019)

    Google Scholar 

  35. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: Proceedings of the 33nd International Conference on Machine Learning. (ICML 2016), vol. 48, pp. 2071–2080 (2016)

    Google Scholar 

  36. Vashishth, S., Sanyal, S., Nitin, V., Talukdar, P.: Composition-based multi-relational graph convolutional networks. In: International Conference on Learning Representations (2020)

    Google Scholar 

  37. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems 30, pp. 5998–6008. Curran Associates, Inc. (2017)

    Google Scholar 

  38. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph Attention Networks. In: International Conference on Learning Representations (2018)

    Google Scholar 

  39. Wang, L., et al.: Link prediction by exploiting network formation games in exchangeable graphs. In: Proceedings of IJCNN 2017, pp. 619–626 (2017). https://ieeexplore.ieee.org/document/7965910/

  40. Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Proceedings of AAAI 2014, pp. 1112–1119. AAAI Press (2014)

    Google Scholar 

  41. Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)

    MATH  Google Scholar 

  42. Xian, Y., Fu, Z., Muthukrishnan, S., de Melo, G., Zhang, Y.: Reinforcement knowledge graph reasoning for explainable recommendation. In: Proceedings of SIGIR 2019, pp. 285–294. ACM, New York (2019)

    Google Scholar 

  43. Xian, Y., et al.: CAFE: coarse-to-fine knowledge graph reasoning for e-commerce recommendation. In: Proceedings of CIKM 2020. ACM (2020)

    Google Scholar 

  44. Xiong, W., Hoang, T., Wang, W.Y.: DeepPath: a reinforcement learning method for knowledge graph reasoning. In: Proceedings of EMNLP 2017. ACL (2017)

    Google Scholar 

  45. Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. CoRR abs/1412.6575 (2014)

    Google Scholar 

  46. Yang, K., Xinyu, K., Wang, Y., Zhang, J., de Melo, G.: Reinforcement learning over knowledge graphs for explainable dialogue intent mining. IEEE Access 8, 85348–85358 (2020). https://ieeexplore.ieee.org/document/9083954

  47. Yun, S., Jeong, M., Kim, R., Kang, J., Kim, H.J.: Graph Transformer networks. Adv. Neural Inform. Process. Syst. 32, 11983–11993 (2019)

    Google Scholar 

Download references

Acknowledgement

We thank Diffbot for their grant support to Rajarshi Bhowmik’s work. We also thank Diffbot and Google for providing the computing infrastructure required for this project.

Author information

Authors and Affiliations

  1. Rutgers University, Piscataway, New Brunswick, NJ, USA

    Rajarshi Bhowmik

  2. Hasso Plattner Institute, University of Potsdam, Potsdam, Germany

    Gerard de Melo

Authors
  1. Rajarshi Bhowmik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerard de Melo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rajarshi Bhowmik .

Editor information

Editors and Affiliations

  1. University of Edinburgh, Edinburgh, UK

    Jeff Z. Pan

  2. University of Liverpool, Liverpool, UK

    Valentina Tamma

  3. University of Bari, Bari, Italy

    Claudia d’Amato

  4. University of California, Santa Barbara, Santa Barbara, CA, USA

    Krzysztof Janowicz

  5. California State University, Long Beach, Long Beach, CA, USA

    Bo Fu

  6. Vienna University of Economics and Business, Vienna, Austria

    Axel Polleres

  7. Rensselaer Polytechnic Institute, Troy, NY, USA

    Oshani Seneviratne

  8. Massachusetts Institute of Technology, Cambridge, MA, USA

    Lalana Kagal

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Bhowmik, R., de Melo, G. (2020). Explainable Link Prediction for Emerging Entities in Knowledge Graphs. In: , et al. The Semantic Web – ISWC 2020. ISWC 2020. Lecture Notes in Computer Science(), vol 12506. Springer, Cham. https://doi.org/10.1007/978-3-030-62419-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-62419-4_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-62418-7

  • Online ISBN: 978-3-030-62419-4

  • eBook Packages: Computer ScienceComputer Science (R0)