Skip to main content
SpringerLink
Log in

Generalizable inductive relation prediction with causal subgraph

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Inductive relation prediction is an important learning task for knowledge graph reasoning that aims to infer new facts from existing ones. Previous graph neural networks (GNNs) based methods have demonstrated great success in inductive relation prediction by capturing more subgraph information. However, they aggregate all reasoning paths which might introduces redundant information. Such redundant information changes with the context of entity and easily outside the training distribution making existing GNN-base methods suffer from poor generalization. In this work, we propose a novel causal knowledge graph reasoning (CKGR) framework for inductive relation prediction task with better generalization. We first take a causal view of inductive relation prediction and construct a structural causal model (SCM) that reveals the relationship between variables. With our assumption, CKGR extracts causal and shortcut subgraphs conditioned on query triplet. Then, we parameter the backdoor adjustment of causality theory by making intervention in representation space. In this way, CKGR can learn stable causal feature and alleviates the confounding effect of shortcut features that are spuriously correlated to relation prediction. Extensive experiments on various tasks with real-world and synthetic datasets demonstrate the effectiveness of CKGR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The datasets are public datasets, and can get from previous work such as https://raw.githubusercontent.com/kkteru/grail/master/data/.

References

  1. Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G.D., Gutierrez, C., Kirrane, S., Gayo, J.E.L., Navigli, R., Neumaier, S., et al.: Knowledge graphs. ACM Computing Surveys (Csur) 54(4), 1–37 (2021)

    Article  Google Scholar 

  2. Wang, H., Zhang, F., Wang, J., Zhao, M., Li, W., Xie, X., Guo, M.: Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 417–426 (2018)

  3. Lin, R., Tang, F., He, C., Wu, Z., Yuan, C., Tang, Y.: Dirs-kg: a kg-enhanced interactive recommender system based on deep reinforcement learning. World Wide Web, 1–23 (2023)

  4. Huang, X., Zhang, J., Li, D., Li, P.: Knowledge graph embedding based question answering. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 105–113 (2019)

  5. Li, X., Hu, S., Zou, L.: Knowledge based natural answer generation via maskedgraph transformer. World Wide Web, 1–21 (2022)

  6. Ji, S., Pan, S., Cambria, E., Marttinen, P., Philip, S.Y.: A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems 33(2), 494–514 (2021)

    Article  MathSciNet  Google Scholar 

  7. Wan, G., Du, B., Pan, S., Wu, J.: Adaptive knowledge subgraph ensemble for robust and trustworthy knowledge graph completion. World Wide Web 23, 471–490 (2020)

    Article  Google Scholar 

  8. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. Advances in Neural Information Processing Systems 26 (2013)

  9. Trouillon, T., Dance, C.R., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Knowledge graph completion via complex tensor factorization. arXiv:1702.06879 (2017)

  10. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2d knowledge graph embeddings. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

  11. Sun, Z., Deng, Z.-H., Nie, J.-Y., Tang, J.: Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv:1902.10197 (2019)

  12. Yang, F., Yang, Z., Cohen, W.W.: Differentiable learning of logical rules for knowledge base reasoning. Advances in Neural Information Processing Systems 30 (2017)

  13. Sadeghian, A., Armandpour, M., Ding, P., Wang, D.Z.: Drum: End-to-end differentiable rule mining on knowledge graphs. Advances in Neural Information Processing Systems 32 (2019)

  14. Teru, K., Denis, E., Hamilton, W.: Inductive relation prediction by subgraph reasoning. In: International Conference on Machine Learning, pp. 9448–9457 (2020). PMLR

  15. Meilicke, C., Fink, M., Wang, Y., Ruffinelli, D., Gemulla, R., Stuckenschmidt, H.: Fine-grained evaluation of rule-and embedding-based systems for knowledge graph completion. In: The Semantic Web-ISWC 2018: 17th International Semantic Web Conference, Monterey, CA, USA, October 8–12, 2018, Proceedings, Part I 17, pp. 3–20 (2018). Springer

  16. Mai, S., Zheng, S., Yang, Y., Hu, H.: Communicative message passing for inductive relation reasoning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 4294–4302 (2021)

  17. Zhu, Z., Zhang, Z., Xhonneux, L.-P., Tang, J.: Neural bellman-ford networks: A general graph neural network framework for link prediction. Advances in Neural Information Processing Systems 34, 29476–29490 (2021)

    Google Scholar 

  18. Zhang, Y., Yao, Q.: Knowledge graph reasoning with relational digraph. In: Proceedings of the ACM Web Conference 2022, pp. 912–924 (2022)

  19. Morris, C., Kriege, N.M., Bause, F., Kersting, K., Mutzel, P., Neumann, M.: Tudataset: A collection of benchmark datasets for learning with graphs. arXiv:2007.08663 (2020)

  20. Wu, Y.-X., Wang, X., Zhang, A., He, X., Chua, T.-S.: Discovering invariant rationales for graph neural networks. arXiv:2201.12872 (2022)

  21. Sui, Y., Wang, X., Wu, J., Lin, M., He, X., Chua, T.-S.: Causal attention for interpretable and generalizable graph classification. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1696– 1705 (2022)

  22. Li, H., Zhang, Z., Wang, X., Zhu, W.: Learning invariant graph representations for out-of-distribution generalization. Advances in Neural Information Processing Systems 35, 11828–11841 (2022)

    Google Scholar 

  23. Fan, S., Wang, X., Mo, Y., Shi, C., Tang, J.: Debiasing graph neural networks via learning disentangled causal substructure. Advances in Neural Information Processing Systems 35, 24934–24946 (2022)

    Google Scholar 

  24. Pearl, J.: Interpretation and identification of causal mediation. Psychological Methods 19(4), 459 (2014)

    Article  MathSciNet  Google Scholar 

  25. Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics: A Primer. John Wiley & Sons, ??? (2016)

  26. Pearl, J., Mackenzie, D.: The Book of Why: the New Science of Cause and Effect. Basic books, ??? (2018)

  27. Galárraga, L.A., Teflioudi, C., Hose, K., Suchanek, F.: Amie: association rule mining under incomplete evidence in ontological knowledge bases. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 413–422 (2013)

  28. Das, R., Dhuliawala, S., Zaheer, M., Vilnis, L., Durugkar, I., Krishnamurthy, A., Smola, A., McCallum, A.: Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. arXiv:1711.05851 (2017)

  29. Shen, Y., Chen, J., Huang, P.-S., Guo, Y., Gao, J.: M-walk: Learning to walk over graphs using monte carlo tree search. Advances in Neural Information Processing Systems 31 (2018)

  30. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv:1609.02907 (2016)

  31. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  32. Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? arXiv:1810.00826 (2018)

  33. Schlichtkrull, M., Kipf, T.N., Bloem, P., Van Den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15, pp. 593–607 (2018). Springer

  34. Vashishth, S., Sanyal, S., Nitin, V., Talukdar, P.: Composition-based multirelational graph convolutional networks. arXiv:1911.03082 (2019)

  35. Xu, X., Feng, W., Jiang, Y., Xie, X., Sun, Z., Deng, Z.-H.: Dynamically pruned message passing networks for large-scale knowledge graph reasoning. arXiv preprint arXiv:1909.11334 (2019)

  36. Zhang, Y., Zhou, Z., Yao, Q., Chu, X., Han, B.: Adaprop: Learning adaptive propagation for graph neural network based knowledge graph reasoning. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3446–3457 (2023)

  37. Magliacane, S., Van Ommen, T., Claassen, T., Bongers, S., Versteeg, P., Mooij, J.M.: Domain adaptation by using causal inference to predict invariant conditional distributions. Advances in Neural Information Processing Systems 31 (2018)

  38. Zhang, D., Zhang, H., Tang, J., Hua, X.-S., Sun, Q.: Causal intervention for weakly-supervised semantic segmentation. Advances in Neural Information Processing Systems 33, 655–666 (2020)

    Google Scholar 

  39. Tang, K., Huang, J., Zhang, H.: Long-tailed classification by keeping the good and removing the bad momentum causal effect. Advances in Neural Information Processing Systems 33, 1513–1524 (2020)

    Google Scholar 

  40. Wang, T., Sridhar, R., Yang, D., Wang, X.: Identifying and mitigating spurious correlations for improving robustness in nlp models. arXiv:2110.07736 (2021)

  41. Tang, R., Kong, D., Huang, L., Xue, H.: Large language models can be lazy learners: Analyze shortcuts in in-context learning. arXiv preprint arXiv:2305.17256 (2023)

  42. Chen, Y., Zhang, Y., Bian, Y., Yang, H., Kaili, M., Xie, B., Liu, T., Han, B., Cheng, J.: Learning causally invariant representations for out-of-distribution generalization on graphs. Advances in Neural Information Processing Systems 35, 22131–22148 (2022)

    Google Scholar 

  43. Lee, N., Yoon, K., Na, G.S., Kim, S., Park, C.: Shift-robust molecular relational learning with causal substructure. arXiv:2305.18451 (2023)

  44. Wu, Q., Zhang, H., Yan, J., Wipf, D.: Handling distribution shifts on graphs: An invariance perspective. arXiv:2202.02466 (2022)

  45. Li, H., Zhang, Z., Wang, X., Zhu, W.: Invariant node representation learning under distribution shifts with multiple latent environments. ACM Transactions on Information Systems 42(1), 1–30 (2023)

    Google Scholar 

  46. Chang, H., Cai, J., Li, J.: Knowledge graph completion with counterfactual augmentation. In: Proceedings of the ACM Web Conference 2023, pp. 2611–2620 (2023)

  47. Pearl, J., et al.: Models, reasoning and inference. Cambridge, UK: CambridgeUniversityPress 19(2), 3 (2000)

    MathSciNet  Google Scholar 

  48. Yang, B., Yih, W.-t., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv:1412.6575 (2014)

  49. Jang, E., Gu, S., Poole, B.: Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016)

  50. Maddison, C.J., Mnih, A., Teh, Y.W.: The concrete distribution: A continuous relaxation of discrete random variables. arXiv:1611.00712 (2016)

Download references

Funding

This work is supported by the National Key Research and Development Program of China (No. 2022YFB3104103).

Author information

Authors and Affiliations

  1. National University of Defense Technology, 109 Deya Road, Changsha, 410003, Hunan, China

    Han Yu, Ziniu Liu, Hongkui Tu, Kai Chen & Aiping Li

Authors
  1. Han Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziniu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongkui Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aiping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yu conducted the creation of model and wrote the manuscript. Liu performed the analysis of the data and reviewed and revised the manuscript. Tu and Chen reviewed and revised the manuscript. Li programed the key project and provided guidance in writing the manuscript.

Corresponding author

Correspondence to Aiping Li.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethics approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, H., Liu, Z., Tu, H. et al. Generalizable inductive relation prediction with causal subgraph. World Wide Web 27, 24 (2024). https://doi.org/10.1007/s11280-024-01264-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11280-024-01264-5

Keywords

Search

Navigation