Abstract
Graph embedding is an advantageous technique for reducing computational costs and effectively using graph information in machine learning tasks like classification, clustering, and link prediction. As a result, it has become a key method in various research areas. However, different embedding methods may be used depending on the variety of graphs available. One of the most commonly used graph types is the heterogeneous graph (HG) or heterogeneous information network (HIN), which presents unique challenges for graph embedding approaches due to its diverse set of nodes and edges. Several methods have been proposed for heterogeneous graph embedding in recent years to overcome these challenges. This paper aims to review the latest techniques used for this purpose, divided into two main parts: the first part describes the fundamental concepts and obstacles in heterogeneous graph embedding, while the second part compares the most critical methods. Finally, the results are summarized, outlining the challenges and opportunities for future directions.
Similar content being viewed by others
Data availability
The data used to support the findings of this study are included within the article.
References
Ahmed A, Shervashidze N, Narayanamurthy S, Josifovski V, Smola AJ (2013) Distributed large-scale natural graph factorization. In: Proceedings of the 22nd international conference on World Wide Web
Alon U, Yahav E (2020) On the bottleneck of graph neural networks and its practical implications. arXiv preprint arXiv:2006.05205
Bian R, Koh YS, Dobbie G, Divoli A (2019) Network embedding and change modeling in dynamic heterogeneous networks. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval
Cai H, Zheng VW, Chang KC-C (2018) A comprehensive survey of graph embedding: problems, techniques, and applications. IEEE Trans Knowl Data Eng 30(9):1616–1637
Cao B, Kong X, Philip SY (2014) Collective prediction of multiple types of links in heterogeneous information networks. In: 2014 IEEE international conference on data mining
Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph representations. In: Proceedings of the AAAI conference on artificial intelligence
Cen Y, Zou X, Zhang J, Yang H, Zhou J, Tang J (2019). Representation learning for attributed multiplex heterogeneous network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining
Chairatanakul N, Liu X, Hoang NT, Murata T (2022) Heterogeneous graph embedding with single-level aggregation and infomax encoding. Mach Learn. https://doi.org/10.1007/s10994-022-06160-5
Chami I, Abu-El-Haija S, Perozzi B, Ré C, Murphy K (2020) Machine learning on graphs: a model and comprehensive taxonomy. arXiv preprint arXiv:2005.03675, 1
Chang Y, Chen C, Hu W, Zheng Z, Zhou X, Chen S (2022) Megnn: Meta-path extracted graph neural network for heterogeneous graph representation learning. Knowl-Based Syst 235:107611
Chang S, Han W, Tang J, Qi G-J, Aggarwal CC, Huang TS (2015) Heterogeneous network embedding via deep architectures. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining
Chen F, Wang Y-C, Wang B, Kuo C-CJ (2020b) Graph representation learning: a survey. APSIPA Trans Signal Inf Process 9:e15
Chen J, Huang F, Peng J (2021) Msgcn: multi-subgraph based heterogeneous graph convolution network embedding. Appl Sci 11(21):9832
Chen K-J, Lu H, Liu Z, Zhang J (2022) Heterogeneous graph convolutional network with local influence. Knowl-Based Syst 236:107699
Chen T, Sun Y (2017) Task-guided and path-augmented heterogeneous network embedding for author identification. In: Proceedings of the tenth ACM international conference on web search and data mining
Chen H, Yin H, Wang W, Wang H, Nguyen QVH, Li X (2018) PME: projected metric embedding on heterogeneous networks for link prediction. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining
Chen Z, Cai T, Chen C, Zheng Z, Ling G (2019) Sine: side information network embedding. In: International conference on database systems for advanced applications
Chen Z, Chen C, Zhang Z, Zheng Z, Zou Q (2019) Variational graph embedding and clustering with laplacian eigenmaps. IJCAI
Chen D, Lin Y, Li W, Li P, Zhou J, Sun X (2020) Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: Proceedings of the AAAI conference on artificial intelligence
Dai Q, Li Q, Tang J, Wang D (2018) Adversarial network embedding. In: Proceedings of the AAAI conference on artificial intelligence
Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
Dong Y, Chawla NV, Swami A (2017) metapath2vec: scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929
Fan H, Zhang F, Wei Y, Li Z, Zou C, Gao Y, Dai Q (2021) Heterogeneous hypergraph variational autoencoder for link prediction. In: IEEE transactions on pattern analysis and machine intelligence
Fang Y, Zhao X, Huang P, Xiao W, de Rijke M (2022) Scalable representation learning for dynamic heterogeneous information networks via metagraphs. ACM Trans Inf Syst (TOIS) 40(4):1–27
Fang Y, Lin W, Zheng VW, Wu M, Chang KC-C, Li X-L (2016) Semantic proximity search on graphs with metagraph-based learning. In: 2016 IEEE 32nd international conference on data engineering (ICDE)
Fu T-Y, Lee W-C, Lei Z (2017) Hin2vec: explore meta-paths in heterogeneous information networks for representation learning. In: Proceedings of the 2017 ACM on conference on information and knowledge management
Fu X, Zhang J, Meng Z, King I (2020) Magnn: metapath aggregated graph neural network for heterogeneous graph embedding. In: Proceedings of the web conference 2020
Gao H, Wang Z, Ji S (2018) Large-scale learnable graph convolutional networks. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining
Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. stat 1050: 10
Guo Z, Wang F, Yao K, Liang J, Wang Z (2022) Multi-scale variational graph autoencoder for link prediction. In: Proceedings of the fifteenth ACM international conference on web search and data mining
Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Adv Neural Inf Process Syst 30
He Y, Song Y, Li J, Ji C, Peng J, Peng H (2019) Hetespaceywalk: a heterogeneous spacey random walk for heterogeneous information network embedding. In: Proceedings of the 28th ACM international conference on information and knowledge management
Hong H, Guo H, Lin Y, Yang X, Li Z, Ye J (2020) An attention-based graph neural network for heterogeneous structural learning. In: Proceedings of the AAAI conference on artificial intelligence
Hu B, Fang Y, Shi C (2019) Adversarial learning on heterogeneous information networks. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining
Hu Z, Dong Y, Wang K, Sun Y (2020) Heterogeneous graph transformer. In: Proceedings of the web conference 2020
Huang Z, Zheng Y, Cheng R, Sun Y, Mamoulis N, Li X (2016) Meta structure: computing relevance in large heterogeneous information networks. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining
Jamali M, Lakshmanan L (2013) Heteromf: recommendation in heterogeneous information networks using context dependent factor models. In: Proceedings of the 22nd international conference on World Wide Web
Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907
Lee S, Park C, Yu H (2019) Bhin2vec: balancing the type of relation in heterogeneous information network. In: Proceedings of the 28th ACM international conference on information and knowledge management
Li L, Duan L, Wang J, He C, Chen Z, Xie G, Deng S, Luo Z (2023) Memory-enhanced transformer for representation learning on temporal heterogeneous graphs. Data Sci Eng 8(2):98–111
Li J, Fu X, Peng H, Wang S, Zhu S, Sun Q, Yu PS, He L (2021) A Robust and generalized framework for adversarial graph embedding. arXiv preprint arXiv:2105.10651
Ling C, Jiang J, Wang J, Thai M, Xue L, Song J, Qiu M, Zhao L (2023) Deep graph representation learning and optimization for influence maximization. arXiv preprint arXiv:2305.02200
Liu Y, Jin M, Pan S, Zhou C, Zheng Y, Xia F, Philip SY (2022) Graph self-supervised learning: a survey. IEEE Trans Knowl Data Eng 35(6):5879–5900
Liu J, Song L, Wang G, Shang X (2023a) Meta-HGT: metapath-aware HyperGraph transformer for heterogeneous information network embedding. Neural Netw 157:65–76
Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision
Liu Y, Zhang Y, Wang Y, Hou F, Yuan J, Tian J, Zhang Y, Shi Z, Fan J, He Z (2023) A survey of visual transformers. In: IEEE transactions on neural networks and learning systems
Lu Y, Shi C, Hu L, Liu Z (2019) Relation structure-aware heterogeneous information network embedding. In: Proceedings of the AAAI conference on artificial intelligence
Luo L, Fang Y, Cao X, Zhang X, Zhang W (2021) Detecting communities from heterogeneous graphs: a context path-based graph neural network model. In: Proceedings of the 30th ACM international conference on information & knowledge management
Mao Q, Liu Z, Liu C, Sun J (2023) Hinormer: representation learning on heterogeneous information networks with graph transformer. In: Proceedings of the ACM web conference 2023
Mavromatis C, Karypis G (2021) HeMI: multi-view embedding in heterogeneous graphs. arXiv preprint arXiv:2109.07008
Mei G, Pan L, Liu S (2022) Heterogeneous graph embedding by aggregating meta-path and meta-structure through attention mechanism. Neurocomputing 468:276–285
Milani Fard A, Bagheri E, Wang K (2019) Relationship prediction in dynamic heterogeneous information networks. In: European conference on information retrieval
Nelson W, Zitnik M, Wang B, Leskovec J, Goldenberg A, Sharan R (2019) To embed or not: network embedding as a paradigm in computational biology. Front Genet 10:381
Nikolentzos G, Meladianos P, Vazirgiannis M (2017) Matching node embeddings for graph similarity. In: Thirty-first AAAI conference on artificial intelligence
Noori A (2022) A new method for detecting influential nodes in social network graphs using deep learning techniques. Karafan Q Sci J 19(1):607–628. https://doi.org/10.48301/kssa.2022.310565.1786
Pan S, Hu R, Fung S-F, Long G, Jiang J, Zhang C (2019) Learning graph embedding with adversarial training methods. IEEE Trans Cybern 50(6):2475–2487
Pan S, Hu R, Long G, Jiang J, Yao L, Zhang C (2018) Adversarially regularized graph autoencoder for graph embedding. arXiv preprint arXiv:1802.04407
Park C, Kim D, Zhu Q, Han J, Yu H (2019) Task-guided pair embedding in heterogeneous network. In: Proceedings of the 28th ACM international conference on information and knowledge management
Park C, Kim D, Han J, Yu H (2020) Unsupervised attributed multiplex network embedding. In: Proceedings of the AAAI conference on artificial intelligence
Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining
Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434
Schlichtkrull M, Kipf TN, Bloem P, Berg RVD, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference
Shi C, Li Y, Zhang J, Sun Y, Philip SY (2016) A survey of heterogeneous information network analysis. IEEE Trans Knowl Data Eng 29(1):17–37
Shi C, Hu B, Zhao WX, Philip SY (2018a) Heterogeneous information network embedding for recommendation. IEEE Trans Knowl Data Eng 31(2):357–370
Shi Y, Zhu Q, Guo F, Zhang C, Han J (2018) Easing embedding learning by comprehensive transcription of heterogeneous information networks. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining
Sun Y, Han J, Yan X, Yu PS, Wu T (2011) Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proc VLDB Endow 4(11):992–1003
Sun X, Zhang W, Wang Z, Lu W (2023) Variational graph embedding for community detection. In: Neural information processing: 29th international conference, ICONIP 2022, Virtual Event, Proceedings, Part VI, Accessed 22–26 Nov 2022
Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G (2016) Complex embeddings for simple link prediction. In: International conference on machine learning
Tu K, Cui P, Wang X, Wang F, Zhu W (2018) Structural deep embedding for hyper-networks. In: Proceedings of the AAAI conference on artificial intelligence
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Proc Syst 30
Velickovic P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017). Graph attention networks. stat 1050: 20
Wang X, Lu Y, Shi C, Wang R, Cui P, Mou S (2020) Dynamic heterogeneous information network embedding with meta-path based proximity. IEEE Trans Knowl Data Eng 34(3):1117–1132
Wang R, Shi C, Zhao T, Wang X, Ye YF (2021a) Heterogeneous information network embedding with adversarial disentangler. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2021.3096231
Wang X, Bo D, Shi C, Fan S, Ye Y, Philip SY (2022) A survey on heterogeneous graph embedding: methods, techniques, applications and sources. IEEE Trans Big Data. https://doi.org/10.1109/TBDATA.2022.3177455
Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining
Wang X, Ji H, Shi C, Wang B, Ye Y, Cui P, Yu PS (2019) Heterogeneous graph attention network. In: The World Wide Web conference
Wang Y, Duan Z, Liao B, Wu F, Zhuang Y (2019) Heterogeneous attributed network embedding with graph convolutional networks. In: Proceedings of the AAAI conference on artificial intelligence
Wang X, Liu N, Han H, Shi C (2021) Self-supervised heterogeneous graph neural network with co-contrastive learning. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining
Wei X, Xu L, Cao B, Yu PS (2017) Cross view link prediction by learning noise-resilient representation consensus. In: Proceedings of the 26th international conference on World Wide Web
Wu Q, Zhao W, Li Z, Wipf DP, Yan J (2022) Nodeformer: A scalable graph structure learning transformer for node classification. Adv Neural Inf Process Syst 35:27387–27401
Xu M (2021) Understanding graph embedding methods and their applications. SIAM Rev 63(4):825–853
Xu L, Wei X, Cao J, Yu PS (2017) Embedding of embedding (EOE) joint embedding for coupled heterogeneous networks. In: Proceedings of the tenth ACM international conference on web search and data mining
Xue H, Yang L, Jiang W, Wei Y, Hu Y, Lin Y (2020) Modeling dynamic heterogeneous network for link prediction using hierarchical attention with temporal rnn. In: Joint European conference on machine learning and knowledge discovery in databases
Xue H, Yang L, Jiang W, Wei Y, Hu Y, Lin Y (2021) Modeling dynamic heterogeneous network for link prediction using hierarchical attention with temporal rnn. In: Machine learning and knowledge discovery in databases: european conference, ECML PKDD 2020, Ghent, Belgium, Proceedings, Part I, Accessed 14–18 Sep 2020
Yang Y, Guan Z, Li J, Zhao W, Cui J, Wang Q (2021) Interpretable and efficient heterogeneous graph convolutional network. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2021.31013564
Yang B, Wang Y (2022) Representation learning in heterogeneous information networks based on hyper adjacency matrix. In: International conference on database systems for advanced applications
Yang L, Xiao Z, Jiang W, Wei Y, Hu Y, Wang H (2020) Dynamic heterogeneous graph embedding using hierarchical attentions. In: European conference on information retrieval
Ye F, Chen C, Zheng Z, Li R-H, Yu JX (2019) Discrete overlapping community detection with pseudo supervision. In: 2019 IEEE international conference on data mining (ICDM)
Yin Y, Ji L-X, Zhang J-P, Pei Y-L (2019) DHNE: network representation learning method for dynamic heterogeneous networks. IEEE Access 7:134782–134792
Yu L, Sun L, Du B, Liu C, Lv W, Xiong H (2022) Heterogeneous graph representation learning with relation awareness. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2022.3160208
Yu W, Zheng C, Cheng W, Aggarwal CC, Song D, Zong B, Chen H, Wang W (2018) Learning deep network representations with adversarially regularized autoencoders. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining
Yun S, Jeong M, Kim R, Kang J, Kim HJ (2019) Graph transformer networks. Adv Neural Inf Process Syst 32
Zhang C, Huang C, Yu L, Zhang X, Chawla NV (2018) Camel: content-aware and meta-path augmented metric learning for author identification. In: Proceedings of the 2018 World Wide Web conference
Zhang D, Yin J, Zhu X, Zhang C (2018) Metagraph2vec: complex semantic path augmented heterogeneous network embedding. In: Pacific-Asia conference on knowledge discovery and data mining
Zhang C, Song D, Huang C, Swami A, Chawla NV (2019) Heterogeneous graph neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining
Zhang C, Swami A, Chawla NV (2019) Shne: representation learning for semantic-associated heterogeneous networks. In: Proceedings of the twelfth ACM international conference on web search and data mining
Zhao J, Wang X, Shi C, Liu Z, Ye Y (2020) Network schema preserving heterogeneous information network embedding. In: International joint conference on artificial intelligence (IJCAI)
Zhao K, Bai T, Wu B, Wang B, Zhang Y, Yang Y, Nie J-Y (2020) Deep adversarial completion for sparse heterogeneous information network embedding. In: Proceedings of the web conference 2020
Zhao J, Wen Q, Sun S, Ye Y, Zhang C (2021) Multi-view self-supervised heterogeneous graph embedding. In: Joint European conference on machine learning and knowledge discovery in databases
Zhou J, Liu L, Wei W, Fan J (2022) Network representation learning: from preprocessing, feature extraction to node embedding. ACM Comput Surv (CSUR) 55(2):1–35
Zhu S, Zhou C, Pan S, Zhu X, Wang B (2019) Relation structure-aware heterogeneous graph neural network. In: 2019 IEEE international conference on data mining (ICDM)
Zhu Z, Fan X, Chu X, Bi J (2020) Hgcn: a heterogeneous graph convolutional network-based deep learning model toward collective classification. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining
Author information
Authors and Affiliations
Contributions
A.N. and M.B. wrote the main manuscript text. A.N., A.B., and K.S. designed, implemented the experiments, performed the analysis, and designed the figures and tables. M.B. and A.B. were involved in planning and supervised the work. All authors discussed the results and commented on the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Noori, A., Balafar, M.A., Bouyer, A. et al. Review of heterogeneous graph embedding methods based on deep learning techniques and comparing their efficiency in node classification. Soc. Netw. Anal. Min. 14, 17 (2024). https://doi.org/10.1007/s13278-023-01178-6
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13278-023-01178-6