Abstract
Knowledge graphs are often incomplete in practice, so link prediction becomes an important problem in developing many downstream applications. Therefore, many knowledge graph embedding models have been proposed to predict the missing links based on known facts. Convolutional neural networks (CNNs) play an essential role due to their excellent performance and parameter efficiency. Previous CNN-based models such as ConvE and KMAE use kernels to capture interactions between embeddings, yet they are limited in quantity. In this paper, we propose a novel neural network-based model named MixER to exploit more additional interactions effectively. Our model incorporates two types of multi-layer perceptions (i.e., channel-mixing and token-mixing), which extract spatial information and channel features. Hence, MixER can seize richer interactions and boost the link prediction performance. Furthermore, we investigate the characteristics of two core components that benefit in capturing additional interactions in diverse regions. Experimental results reveal that MixER outperforms state-of-the-art models in the branch of CNNs on three benchmark datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Miller, G.A.: WordNet: a lexical database for English. Commun. ACM (1995)
Suchanek, F.M., Kasneci, G., Weikum, G.: Yago: a core of semantic knowledge. In: ACM (2007)
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: ACM (2008)
Tolstikhin, I.O., Houlsby, N., et al.: MLP-Mixer: an all-MLP architecture for vision. In: NeurIPS (2021)
Toutanova, K., Chen, D.: Observed versus latent features for knowledge base and text inference. In: ACL (2015)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: AAAI (2018)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS (2013)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: AAAI (2014)
Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: ACL (2015)
Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: RotatE: knowledge graph embedding by relational rotation in complex space. In: ICLR (2019)
Le, T., Huynh, N., Le, B.: Link prediction on knowledge graph by rotation embedding on the hyperplane in the complex vector space. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds.) ICANN 2021. LNCS, vol. 12893, pp. 164–175. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86365-4_14
Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., Phung, D.: A novel embedding model for knowledge base completion based on convolutional neural network. In: NAACL-HLT (2018)
Vu, T., Nguyen, T.D., Nguyen, D.Q., Phung, D.: A capsule network-based embedding model for knowledge graph completion and search personalization. In: ACL (2019)
Huang, J., Zhang, T., Zhu, J., Yu, W., Tang, Y., He, Y.: A deep embedding model for knowledge graph completion based on attention mechanism. In: NCA* (2021)
Yang, B., Yih, W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. In: ICLR (2015)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É, Bouchard, G.: Complex embeddings for simple link prediction. In: PMLR (2016)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Cai, L., Wang, W.Y.: KBGAN: adversarial learning for knowledge graph embeddings. In: ACL (2018)
Bansal, T., Juan, D.C., Ravi, S., McCallum, A.: A2N: attending to neighbors for knowledge graph inference. In: ACL (2019)
Jiang, D., Wang, R., Yang, J., Xue, L.: Kernel multi-attention neural network for knowledge graph embedding. In: KBS (2021)
Allen, C., Balažević, I., Hospedales, T.: Interpreting knowledge graph relation representation from word embeddings. In: ICLR (2021)
Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: ICML (2011)
Nickel, M., Tresp, V., Kriegel, H.P.: A comprehensive overview of knowledge graph completion. In: KBS (2022)
Hogan, A., et al.: Knowledge graphs. In: ACM Computing Surveys (2021)
Acknowledgement
This research is supported by the research funding from the Faculty of Information Technology, University of Science, Ho Chi Minh city, Vietnam.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Le, T., Pham, A., Chung, T., Nguyen, T., Nguyen, T., Le, B. (2023). MixER: MLP-Mixer Knowledge Graph Embedding for Capturing Rich Entity-Relation Interactions in Link Prediction. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13936. Springer, Cham. https://doi.org/10.1007/978-3-031-33377-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-33377-4_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33376-7
Online ISBN: 978-3-031-33377-4
eBook Packages: Computer ScienceComputer Science (R0)