Skip to main content

Advertisement

SpringerLink
Log in

Geometry-based anisotropy representation learning of concepts for knowledge graph embedding

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The entities in the knowledge graphs are generally categorized into concepts and instances, where each concept is used to represent the abstraction of a set of instances with common properties. Most previous Knowledge Graph Embedding methods tend to treat them in the same way by projecting them into low-dimension space as vector points without explicitly distinguishing them, therefore ignoring the potential specification of concept. Some recent studies address this problem by modeling each concept as a sphere rather than a vector point. However, the isotropy of the sphere is less capable of modeling the semantic abstraction of concepts, as well as the complex relations between concepts and instances. To solve this problem, we propose to model concepts using geometric shapes with anisotropy to enrich the representation power of concepts. Two algorithms, named as TransEllipsoid and TransCuboid, are presented to project each concept as an ellipsoid and a cuboid in embedding space respectively. The anisotropy of concept embedding is learned by allowing the length of the axes of the ellipsoid or the edges of the cuboid to vary across different dimensions. Experimental results on three real-world datasets show that modeling the anisotropy of concept embedding would significantly benefit not only the learned representations of concepts but also the corresponding instances. The visualization of embedding results reveals human-intuitive relative positions between concepts and instances and provides potential interpretability for the transitivity of isA relations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availabilty

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Zhang N, Deng S, Sun Z, et al (2019) Long-tail relation extraction via knowledge graph embeddings and graph convolution networks. In: proceedings of the 2019 conference of the north american chapter of the association for computational linguistics : human language technologies, vol 1 (Long and short papers). Association for computational linguistics. Minnesota, Minneapolis, pp 3016–3025, https://doi.org/10.18653/v1/N19-1306

  2. Chen Y, Wu L, Zaki MJ (2019) Bidirectional Attentive Memory Networks for Question Answering over Knowledge Bases. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics : human language technologies, vol 1 (Long and short papers). Association for computational linguistics. Minnesota, Minneapolis, pp 2913–2923, https://doi.org/10.18653/v1/N19-1299

  3. He H, Balakrishnan A, Eric M, et al (2017) Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. In: Proceedings of the 55th annual meeting of the association for computational linguistics (vol 1 : long papers). Association for computational linguistics, Vancouver, Canada, pp 1766-1776, https://doi.org/10.18653/v1/P17-1162

  4. Bordes A, Usunier N, Garcia-Duran A, et al (2013) Translating embeddings for modeling multi-relational data. In: Burges C J, Bottou L, Welling M (eds) Advances in neural information processing systems, vol 26. Curran Associates, Inc

  5. Lv X, Hou L, Li J, et al (2018) Differentiating concepts and instances for knowledge graph embedding. In: Proceedings of the 2018 conference on empirical methods in natural language processing. Association for computational linguistics, Brussels, Belgium, pp 1971-1979. https://doi.org/10.18653/v1/D18-1222

  6. Ji S, Pan S, Cambria E, et al (2021) A survey on knowledge graphs : representation, acquisition, and applications. IEEE Trans Neural Netw Learn Syst 33(2):494–514. https://doi.org/10.1109/TNNLS.2021.3070843

    Article  MathSciNet  Google Scholar 

  7. Liu H, Wang Y, Wu F et al (2019) REKER: relation extraction with knowledge of entity and relation. In: Tang J, Kan MY, Zhao D (eds) Natural language processing and chinese computing, Springer International Publishing, Cham, pp 90-102

  8. Zhang H, Liu Z, Xiong C, et al (2020) Grounded conversation generation as guided traverses in commonsense knowledge graphs. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, pp 2031-2043. https://doi.org/10.18653/v1/2020.acl-main.184

  9. Miller GA, Beckwith R, Fellbaum C, et al (1990) Introduction to WordNet : an On-line lexical database*. Int J Lexicogr 3(4):235–244. https://doi.org/10.1093/ijl/3.4.235

    Article  Google Scholar 

  10. Suchanek FM, Kasneci G, Weikum G (2007) Yago : a core of semantic knowledge. In: 16th international world wide web conference, Banff, AB, Canada, pp 697–706

  11. Hao J, Chen M, Yu W, et al (2019) Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. association for computing machinery, New York, NY, USA, KDD ’19, pp 1709–1719 event-place : Anchorage, AK, USA. https://doi.org/10.1145/3292500.3330838

  12. Zhou J, Wang P, Pan Z, et al (2020) JECI : a joint knowledge graph embedding model for concepts and instances. In: Wang X, Lisi FA, Xiao G (eds) Semantic Technology, vol 12032. Springer International Publishing, Cham, pp 82-98. https://doi.org/10.1007/978-3-030-41407-8_6

  13. Yu Y, Xu Z, Lv Y, et al (2019) TransFG : a fine-grained model for knowledge graph embedding. In: Ni W, Wang X, Song W (eds) Web Information Systems and Applications, Springer International Publishing, Cham, pp 455-466. https://doi.org/10.1007/978-3-030-30952-7_45

  14. Fan M, Zhou Q, Chang E, et al (2014) Transition-based knowledge graph embedding with relational mapping properties. In: Proceedings of the 28th pacific asia conference on language, information and computing. department of linguistics, Chulalongkorn University, Phuket,Thailand, pp 328-337

  15. Wang Z, Zhang J, Feng J, et al (2014) Knowledge graph embedding by translating on hyperplanes. In: Brodley CE, Stone P (eds) Proceedings of the twenty-eighth AAAI conference on artificial intelligence. AAAI Press, pp 1112-1119

  16. Lin Y, Liu Z, Sun M, et al (2015) Learning entity and relation embeddings for knowledge graph completion. In: Bonet B, Koenig S (eds) Proceedings of the twenty-ninth AAAI conference on artificial intelligence. AAAI Press, pp 2181-2187

  17. Ji G, He S, Xu L, et al (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (vol 1: long papers). Association for computational linguistics, Beijing, China, pp 687-696. https://doi.org/10.3115/v1/P15-1067

  18. Yang S, Tian J, Zhang H et al (2019) TransMS : knowledge graph embedding for complex relations by multidirectional semantics. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI-19. International joint conferences on artificial intelligence organization, pp 1935–1942 https://doi.org/10.24963/ijcai.2019/268

  19. Nayyeri M, Cil GM, Vahdati S, et al (2021) Trans4E : link prediction on scholarly knowledge graphs. Neurocomputing 461:530–542. https://doi.org/10.1016/j.neucom.2021.02.100

    Article  Google Scholar 

  20. Zhang Z, Cai J, Zhang Y et al (2020) Learning hierarchy-aware knowledge graph embeddings for link prediction. In: Proceedings of the AAAI conference on artificial intelligence, pp 3065–3072. https://doi.org/10.1609/aaai.v34i03.5701

  21. Yang B, Yih WT, He X et al (2015) Embedding entities and relations for learning and inference in knowledge bases. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations

  22. Nickel M, Rosasco L, Poggio TA (2016) Holographic embeddings of knowledge graphs. In: Schuurmans D, Wellman MP (eds) Proceedings of the thirtieth AAAI conference on artificial intelligence. AAAI Press, pp 1955–1961

  23. Trouillon T, Welbl J, Riedel S et al (2016) Complex embeddings for simple link prediction. In: Proceedings of the 33rd international conference on international conference on machine learning - vol 48. JMLR.org, ICML’16, p 2071–2080

  24. Zhang W, Paudel B, Zhang W et al (2019) Interaction embeddings for prediction and explanation in knowledge graphs. In: Proceedings of the twelfth ACM international conference on web search and data mining, pp 96–104. https://doi.org/10.1145/3289600.3291014

  25. Shi B, Weninger T (2017) ProjE: embedding projection for knowledge graph completion. In: Singh S, Markovitch S (eds) Proceedings of the thirty-first AAAI conference on artificial intelligence. AAAI Press, pp 1236-1242

  26. Dettmers T, Minervini P, Stenetorp P, et al (2018) Convolutional 2D knowledge graph embeddings. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the thirty-second AAAI conference on artificial intelligence. AAAI Press, pp 1811-1818

  27. Vashishth S, Sanyal S, Nitin V et al (2020) InteractE : improving convolution-based knowledge graph embeddings by increasing feature interactions. In: The thirty-fourth AAAI conference on artificial intelligence. AAAI Press, pp 3009–3016

  28. Balažević I, Allen C, Hospedales TM (2019) Hypernetwork knowledge graph embeddings. In: Tetko IV, Kůrková V, Karpov P (eds) Artificial neural networks and machine learning – ICANN, workshop and special sessions, Springer International Publishing, Cham, pp 553-565

  29. Vashishth S, Sanyal S, Nitin V, et al (2020) Composition-based multi-relational graph convolutional networks. In: International conference on learning representations. OpenReview.net

  30. Balazevic I, Allen C, Hospedales T, et al (2019) Multi-relational poincaré graph embeddings. In: Wallach H, Larochelle H, Beygelzimer A (eds) Advances in Neural Information Processing Systems. vol 32. Curran Associates, Inc

  31. Pan Z, Wang P (2021) Hyperbolic hierarchy-aware knowledge graph embedding for link prediction. In: Findings of the association for computational linguistics: EMNLP. Association for Computational Linguistics, Punta Cana, Dominican Republic, pp 2941-2948. https://doi.org/10.18653/v1/2021.findings-emnlp.251

  32. Wang K, Liu Y, Lin D, et al (2021) Hyperbolic geometry is not necessary : lightweight euclidean-based models for low-dimensional knowledge graph embeddings. In: Findings of the association for computational linguistics : EMNLP. Association for Computational Linguistics, Punta Cana, Dominican Republic, pp 464-474. https://doi.org/10.18653/v1/2021.findings-emnlp.42

  33. Chen M, Tian Y, Chen X et al (2018) On2Vec : embedding-based relation prediction for ontology population. In: Proceedings of the 2018 SIAM international conference on data mining (SDM), pp 315–323. https://doi.org/10.1137/1.9781611975321.36

  34. Gutiérrez-Basulto V, Schockaert S (2018) From knowledge graph embedding to ontology embedding? an analysis of the compatibility between vector space representations and rules. In: Thielscher M, Toni F, Wolter F (eds) Principles of knowledge representation and reasoning: proceedings of the sixteenth international conference. AAAI Press, pp 379-388

  35. Diaz GI, Fokoue A, Sadoghi M, et al (2018) EmbedS : scalable, ontology-aware graph embeddings. In: Böhlen MH, Pichler R, May N (eds) Proceedings of the 21st international conference on extending database technology. OpenProceedings.org, pp 433-436. https://doi.org/10.5441/002/edbt.2018.40

  36. Gao H, Zheng X, Li W, et al (2019) Cosine-based embedding for completing schematic knowledge. In: Tang J, Kan MY, Zhao D (eds) Natural language processing and chinese computing, Springer International Publishing, Cham, pp 249-261. https://doi.org/10.1007/978-3-030-32233-5_20

  37. Qiu J, Wang S, et al (2020) Learning the concept embeddings of ontology. In: Yang X, Wang CD, Islam MS (eds) Advanced data mining and applications, Springer International Publishing, Cham, pp 127-134. https://doi.org/10.1007/978-3-030-65390-3_10

  38. Hu Z, Huang P, Deng Y, et al (2015) Entity hierarchy embedding. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (vol 1 : long papers). Association for Computational Linguistics, Beijing, China, pp 1292-1300. https://doi.org/10.3115/v1/P15-1125

  39. Guo S, Wang Q, Wang B et al (2017) SSE : semantically smooth embedding for knowledge graphs. IEEE Trans Knowl Data Eng 29(4):884–897. https://doi.org/10.1109/TKDE.2016.2638425, publisher : IEEE Computer Society

    Article  Google Scholar 

  40. Guan N, Song D, Liao L (2019) Knowledge graph embedding with concepts. Knowl-Based Syst 164:38–44. https://doi.org/10.1016/j.knosys.2018.10.008, publisher: Elsevier BV

  41. Xiang Y, Zhang Z, Chen J, et al (2021) OntoEA : ontology-guided entity alignment via joint knowledge graph embedding. In: Findings of the association for computational linguistics : ACL-IJCNLP 2021. Association for Computational Linguistics, Online, pp 1117-1128. https://doi.org/10.18653/v1/2021.findings-acl.96

  42. Dong Y, Wang L, Xiang J et al (2022) Modeling IsA relations via box structure for knowledge graph embedding. In: Gama J, Li T, Yu Y (eds) Advances in knowledge discovery and data mining, lecture notes in computer science, vol 13281.Springer, pp 303-315. https://doi.org/10.1007/978-3-031-05936-0_24

  43. Socher R, Chen D, Manning CD et al (2013) Reasoning with neural tensor networks for knowledge base completion. In: Burges CJ, Bottou L, Welling M (eds) Advances in neural information processing systems, vol 26. Curran Associates, Inc

  44. Jenatton R, Roux N, Bordes A, et al (2012) A latent factor model for highly multi-relational data. In: Pereira F, Burges CJ, Bottou L (eds) Advances in neural information processing systems, vol 25. Curran Associates, Inc

  45. Bordes A, Weston J, Collobert R, et al (2011) Learning structured embeddings of knowledge bases. In: Burgard W, Roth D (eds) Proceedings of the twenty-fifth AAAI conference on artificial intelligence. AAAI Press

  46. Li W, Peng R, Li Z (2022) Improving knowledge graph completion via increasing embedding interactions. Appl Intell 52(8):9289–9307. https://doi.org/10.1007/s10489-021-02947-6

    Article  Google Scholar 

  47. Maaten LVD, Hinton G (2008) Visualizing Data using t-SNE. J Mach Learn Res 9(86):2579–2605

    MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the Guangdong Province Science and Technology Project 2021A0505080015 and KeyArea Research and Development Program of Guangdong Province under Grant Agreement No 2020B0101130013.

Author information

Authors and Affiliations

  1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Jibin Yu & Zheng Hu

  2. Key Laboratory of Universal Wireless Communications, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Chunhong Zhang, Yang Ji, Dongjun Fu & Xueyu Wang

Authors
  1. Jibin Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunhong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zheng Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dongjun Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xueyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zheng Hu.

Ethics declarations

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, J., Zhang, C., Hu, Z. et al. Geometry-based anisotropy representation learning of concepts for knowledge graph embedding. Appl Intell 53, 19940–19961 (2023). https://doi.org/10.1007/s10489-023-04528-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04528-1

Keywords

Search

Navigation