Skip to main content
SpringerLink
Log in

W-KG2Vec: a weighted text-enhanced meta-path-based knowledge graph embedding for similarity search

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Recently, similar entity searching over knowledge graph (KG) has gained much attentions by researchers. However, in rich-semantic KGs with multi-typed entities and relations, also known as heterogeneous information network, relevant entity search is considered as a challenging task due to the ambiguity as well as complexity of user’s queries in realistic applications, such as QA chatbot and KG-based information retrieval. In this paper, we propose a novel approach, called W-KG2Vec which enables to automatically learn the semantic representations of entities in KG by applying the meta-path. The proposed W-KG2Vec is a meta-path-specific model which supports to evaluate both semantic relations as well as the text-based similarity between entities. The combination of text- and structure-based embedding mechanism of W-KG2Vec is promising to achieve better representations of entities in given KGs for handling complex user’s queries. To effectively learn the sequential textual representations of entities’ descriptions, we propose a combination of BERT pre-trained model with LTSM encoder, called BERT-Text2Vec. Then, the text-based similarity between entities is used to leverage our weighted meta-path-based random walk mechanism in W-KG2Vec model. Extensive experiences on real-world KGs (YAGO and Freebase) demonstrate the effectiveness of our proposed model against recent state-of-the-art KG embedding baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. Adv Neural Inf Process Syst 2:2787–2795

    Google Scholar 

  2. Bordes A, Weston J, Usunier N (2014) Open question answering with weakly supervised embedding models. In: Joint European conference on machine learning and knowledge discovery in databases

  3. Bordes A, Glorot X, Weston J, Bengio Y (2012) Joint learning of words and meaning representations for open-text semantic parsing. Artif Intell Stat, pp 127–135

  4. Bordes A, Chopra S, Weston J (2014) Question answering with subgraph embeddings. arXiv preprint https://arxiv.org/abs/1406.3676.

  5. Bordes A, Weston J, Collobert R, Bengio Y (2011) Learning structured embeddings of knowledge bases. In: Twenty-fifth AAAI conference on artificial intelligence

  6. Cao X, Shi C, Zheng Y, Ding J, Li X, Wu B (2018) A heterogeneous information network method for entity set expansion in knowledge graph. In Pacific-Asia conference on knowledge discovery and data mining

  7. Conneau A, Kiela D, Schwenk H, Barrault L, Bordes A (2017) Supervised learning of universal sentence representations from natural language inference data. arXiv preprint https://arxiv.org/abs/1705.02364

  8. Dettmers T, Minervini P, Stenetorp P, Riedel S (2017) Convolutional 2d knowledge graph embeddings. arXiv preprint https://arxiv.org/abs/1707.01476

  9. Fang Y, Wang H, Zhao L, Yu F, Wang C (2020) Dynamic knowledge graph based fake-review detection. Appl Intell 50:4281–4295

    Article  Google Scholar 

  10. Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining

  11. Han B, Chen L, Tian X (2018) Knowledge based collection selection for distributed information retrieval. Inf Process Manage 54(1):116–128

    Article  Google Scholar 

  12. Han X, Liu Z, Sun M (2016) Joint representation learning of text and knowledge for knowledge graph completion. arXiv preprint https://arxiv.org/abs/1611.04125

  13. He S, Liu K, Ji G, Zhao J (2015) Learning to represent knowledge graphs with Gaussian embedding. In: Proceedings of the 24th ACM international on conference on information and knowledge management

  14. Hussein R, Yang D, Cudré-Mauroux P (2018) Are meta-paths necessary? revisiting heterogeneous graph embeddings. In: Proceedings of the 27th ACM international conference on information and knowledge management

  15. Li B, Pi D (2020) Network representation learning: a systematic literature review. Neural Comput Appl 32:1–33

    Article  Google Scholar 

  16. Lin X, Liang Y, Giunchiglia F, Feng X, Guan R (2019) Relation path embedding in knowledge graphs. Neural Comput Appl 31(9):5629–5639

    Article  Google Scholar 

  17. Lin J, Zhao Y, Huang W, Liu C, Pu H (2020) Domain knowledge graph-based research progress of knowledge representation. Neural Comput Appl 33:1–10

    Google Scholar 

  18. Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Twenty-ninth AAAI conference on artificial intelligence

  19. Liu W, Zhou P, Zhao Z, Wang Z, Ju Q, Deng H, Wang P (2020) K-BERT: enabling language representation with knowledge graph. In: AAAI

  20. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint https://arxiv.org/abs/1301.3781

  21. Nguyen DQ, Sirts K, Qu L, Johnson M (2016) Stranse: a novel embedding model of entities and relationships in knowledge bases. arXiv preprint https://arxiv.org/abs/1606.08140

  22. Nickel M, Tresp V, Kriegel HP (2011) A three-way model for collective learning on multi-relational data. In ICML

  23. Nie A, Bennett E, Goodman N (2019) DisSent: learning sentence representations from explicit discourse relations. In Proceedings of the 57th annual meeting of the association for computational linguistics

  24. Pham PDP (2019) W-MetaPath2Vec: the topic-driven meta-path-based model for large-scaled content-based heterogeneous information network representation learning. Expert Syst Appl 123:328–344

    Article  Google Scholar 

  25. Pham P, Do P (2020) W-Metagraph2Vec: a novel approval of enriched schematic topic-driven heterogeneous information network embedding. Int J Mach Learn Cybern 11:1–20

    Article  Google Scholar 

  26. Pham P, Do P (2020) W-Com2Vec: A topic-driven meta-path-based intra-community embedding for content-based heterogeneous information network. Intell Data Anal 24(5):1207–1233

    Article  Google Scholar 

  27. Pham P, Do P, Ta CD (2018) W-PathSim: novel approach of weighted similarity measure in content-based heterogeneous information networks by applying LDA topic modeling. In: Asian conference on intelligent information and database systems

  28. Socher R, Chen D, Manning CD, Ng A (2013) Reasoning with neural tensor networks for knowledge base completion. In: Advances in neural information processing systems

  29. Sun Z, Deng ZH, Nie JY, Tang J (2019) Rotate: knowledge graph embedding by relational rotation in complex space. arXiv preprint https://arxiv.org/abs/1902.10197

  30. Sun Y, Han J, Yan X, Yu PS, Wu T (2011) Pathsim: meta path-based top-k similarity search in heterogeneous information networks. In: Proceedings of the VLDB endowment

  31. Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Toutanova, Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 conference on empirical methods in natural language processing

  32. Wang H, Jiang S, Yu Z (2020) Modeling of complex internal logic for knowledge base completion. Appl Intell 50:3336–3349

    Article  Google Scholar 

  33. Wang Q, Mao Z, Wang B, Guo L (2017) Knowledge graph embedding: A survey of approaches and applications. IEEE Trans Knowl Data Eng 29(12):2724–2743

    Article  Google Scholar 

  34. Wang X, He X, Cao Y, Liu M, Chua TS (2019) Kgat: knowledge graph attention network for recommendation. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining

  35. Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Twenty-Eighth AAAI conference on artificial intelligence

  36. Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph and text jointly embedding. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)

  37. Wu Y, Pan J, Lu P, Lin K, Yu Z (2017) Knowledge graph embedding translation based on constraints. J Inf Hiding Multimedia Signal Process Ubiquitous Int 8(5):1119–1131

    Google Scholar 

  38. Xiao H, Huang M, Zhu X (2016) TransG: A generative model for knowledge graph embedding. In: Proceedings of the 54th annual meeting of the association for computational linguistics (Volume 1: Long Papers)

  39. Xu J, Chen K, Qiu X, Huang X (2016) Knowledge graph representation with jointly structural and textual encoding. arXiv preprint https://arxiv.org/abs/1611.08661

  40. Yao L, Mao C, Luo Y (2019) KG-BERT: BERT for knowledge graph completion. arXiv preprint https://arxiv.org/abs/1909.03193

  41. Zhang D, Yuan B, Wang D, Liu R (2015) Joint semantic relevance learning with text data and graph knowledge. In: Proceedings of the 3rd workshop on continuous vector space models and their compositionality

  42. Zhong H, Zhang J, Wang Z, Wan H, Chen Z (2015) Aligning knowledge and text embeddings by entity descriptions. In: Proceedings of the 2015 conference on empirical methods in natural language processing

Download references

Acknowledgements

This research is funded by Vietnam National University HoChiMinh City (VNU-HCM) under grant number DS2020-26-01.

Author information

Authors and Affiliations

  1. University of Information Technology (UIT), VNU-HCM, Ho Chi Minh City, Vietnam

    Phuc Do & Phu Pham

Authors
  1. Phuc Do
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Phu Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Phuc Do.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Do, P., Pham, P. W-KG2Vec: a weighted text-enhanced meta-path-based knowledge graph embedding for similarity search. Neural Comput & Applic 33, 16533–16555 (2021). https://doi.org/10.1007/s00521-021-06252-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06252-8

Keywords

Search

Navigation