Abstract
In recent years, temporal knowledge graph reasoning has been a critical task in natural language processing. Temporal knowledge graphs store temporal facts that model dynamic relationships or interactions between entities along the timeline. Most existing temporal knowledge graph reasoning methods need a large number of training instances (i.e. support entity facts) for each relation. However, the same as traditional knowledge graphs, temporal knowledge graphs also exhibit long-tailed relational frequency distribution, in which most relationships often do not have many support entity pairs for training. To address this problem, in this paper, we propose a one-shot learning framework (OSLT) applied to temporal knowledge graph link prediction, which aims to predict new relational facts with only one support instance. Specifically, OSLT employs an fact encoder based on Temporal Convolutional Network to encode historical information and model connection of facts at the same timestamp by the aggregator with an attention mechanism. After that, a matching network is employed to compute the similarity score between support fact and query fact. Experiments show that the proposed method outperforms the state-of-the-art baselines on two benchmark datasets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abboud R, Ceylan I, Lukasiewicz T, Salvatori T (2020) Boxe: A box embedding model for knowledge base completion. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H (eds) Advances in Neural Information Processing Systems, vol 33. Curran Associates, Canada, pp 9649–9661
Bai S, Kolter JZ, Koltun V (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR arXiv:1803.01271
Barracchia EP, Pio G, Bifet A, Gomes HM, Pfahringer B, Ceci M (2022) LP-ROBIN: link prediction in dynamic networks exploiting incremental node embedding. Inf. Sci. 606:702–721. https://doi.org/10.1016/j.ins.2022.05.079
Bordes A, Usunier N, García-Durán A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013, Lake Tahoe, pp. 2787–2795
Chen Y, Kang Y, Chen Y, Wang Z (2020) Probabilistic forecasting with temporal convolutional neural network. Neurocomputing 399:491–501. https://doi.org/10.1016/j.neucom.2020.03.011
Chen J, Chen J, Yu Z (2019) Incorporating structured commonsense knowledge in story completion. Proceedings of the AAAI Conference on Artificial Intelligence, 6244–6251
Chen M, Zhang W, Zhang W, Chen Q, Chen H (2019) Meta relational learning for few-shot link prediction in knowledge graphs. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 4216–4225. Association for Computational Linguistics, Hong Kong
Dasgupta SS, Ray SN, Talukdar PP (2018) Hyte: Hyperplane-based temporally aware knowledge graph embedding. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2001–2011. Association for Computational Linguistics, Brussels
Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning, ICML 2017. Proceedings of Machine Learning Research, vol. 70, pp. 1126–1135. PMLR, Sydney
García-Durán A, Dumancic S, Niepert M (2018) Learning sequence encoders for temporal knowledge graph completion. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4816–4821. Association for Computational Linguistics, Brussels
Goyal P, Chhetri SR, Canedo A (2020) dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowl. Based Syst. 187. https://doi.org/10.1016/j.knosys.2019.06.024
Hao J, Ju CJ-, Chen M, Sun Y, Zaniolo C, Wang W (2020) Bio-joie: Joint representation learning of biological knowledge bases. In: International Conference on Bioinformatics, Computational Biology and Health Informatics, pp. 42–14210. ACM, USA
He H, Balakrishnan A, Eric M, Liang P (2017) Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. CoRR arXiv:1704.07130
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE Computer Society, Las Vegas
Jiang T, Liu T, Ge T, Sha L, Chang B, Li S, Sui Z (2016) Towards time-aware knowledge graph completion. In: Calzolari N, Matsumoto Y, Prasad R (eds.) COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, pp. 1715–1724. ACL, Osaka
Jin W, Zhang C, Szekely PA, Ren X (2019) Recurrent event network for reasoning over temporal knowledge graphs. CoRR
Koren Y, Bell RM, Volinsky C (2018) Matrix factorization techniques for recommender systems. Comput. Inf. Sci. 11(2):1–10
Leblay J, Chekol MW (2018) Deriving validity time in knowledge graph. In: Champin P, Gandon F, Lalmas M, Ipeirotis PG (eds) Companion of the The Web Conference 2018. ACM, Lyon, pp 1771–1776
Leetaru K, Schrodt PA (2013) Gdelt: Global data on events, location, and tone, 1979–2012. In: ISA Annual Convention, vol. 2, pp. 1–49. Citeseer
Liu Z, Xiong C, Sun M, Liu Z (2018) Entity-duet neural ranking: Understanding the role of knowledge graph semantics in neural information retrieval. In: Gurevych I, Miyao Y (eds.) Proceedings of the 56th Annual Meeting of Th Association for Computational Linguistics, pp. 2395–2405. ACL, Melbourne
Messner J, Abboud R, Ceylan İİ (2022) Temporal knowledge graph completion using box embeddings. Proceedings of the AAAI Conference on Artificial Intelligence 36(7):7779–7787. https://doi.org/10.1609/aaai.v36i7.20746
Mirtaheri M, Rostami M, Ren X, Morstatter F, Galstyan A (2021) One-shot learning for temporal knowledge graphs. In: Chen D, Berant J, McCallum A, Singh S (eds) 3rd Conference on Automated Knowledge Base Construction. AKBC, Virtual
Ravi S, Larochelle H (2017) Optimization as a model for few-shot learning. In: 5th International Conference on Learning Representations, ICLR 2017. OpenReview.net, Toulon
Sadeghian A, Armandpour M, Colas A, Wang DZ (2021) Chronor: Rotation based temporal knowledge graph embedding. Proceedings of the AAAI Conference on Artificial Intelligence 35(7):6471–6479. https://doi.org/10.1609/aaai.v35i7.16802
Salimans T, Kingma DP (2016) Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In: Lee DD, Sugiyama M, von Luxburg U, Guyon I, Garnett R (eds.) Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems, Barcelona, p. 901
Schlichtkrull MS, Kipf TN, Bloem P, van den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. The Semantic Web - 15th International Conference, ESWC 2018, vol 10843. Springer, Heraklion, pp 593–607
Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: Guyon I, von Luxburg U, Bengio S, Wallach HM, Fergus R, Vishwanathan SVN, Garnett R (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, pp. 4077–4087
Sun Z, Deng Z, Nie J, Tang J (2019) Rotate: Knowledge graph embedding by relational rotation in complex space. In: 7th International Conference on Learning Representations, ICLR 2019. OpenReview.net, New Orleans
Trivedi R, Dai H, Wang Y, Song L (2017) Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 3462–3471. PMLR, Sydney
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, pp. 5998–6008
Vinyals O, Blundell C, Lillicrap T, Kavukcuoglu K, Wierstra D (2016) Matching networks for one shot learning. In: Lee DD, Sugiyama M, von Luxburg U, Guyon I, Garnett R (eds.) Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems, Barcelona, pp. 3630–3638
Xiong W, Yu M, Chang S, Guo X, Wang WY (2018) One-shot relational learning for knowledge graphs. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1980–1990. Association for Computational Linguistics, Brussels
Yang B, Yih W, He X, Gao J, Deng L (2015) Embedding entities and relations for learning and inference in knowledge bases. In: Bengio Y, LeCun Y (eds) 3rd International Conference on Learning Representations. ICLR, San Diego
Zhang C, Yao H, Huang C, Jiang M, Li Z, Chawla NV (2020) Few-shot knowledge graph completion. Proceedings of the AAAI Conference on Artificial Intelligence 34(03):3041–3048
Zhu C, Chen M, Fan C, Cheng G, Zhang Y (2021) Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks. Proceedings of the AAAI Conference on Artificial Intelligence 35:4732–4740. https://doi.org/10.1609/aaai.v35i5.16604
Acknowledgements
This work is supported by the National Natural Science Foundation of China (61906030), the Science and Technology Project of Liaoning Province (2021JH2/10300064), the Youth Science and Technology Star Support Program of Dalian City (2021RQ057) and the Fundamental Research Funds for the Central Universities (DUT22YG241).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.
Additional information
Responsible editor: Michelangelo Ceci
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ma, R., Mei, B., Ma, Y. et al. One-shot relational learning for extrapolation reasoning on temporal knowledge graphs. Data Min Knowl Disc 37, 1591–1608 (2023). https://doi.org/10.1007/s10618-023-00935-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-023-00935-7