Abstract
Graph Neural Networks (GNNs) have achieved excellent performance of graph representation learning and attracted plenty of attentions in recent years. Most of GNNs aim to learn embedding vectors of the homogeneous graph which only contains single type of nodes and edges. However, the entities and their interactions in real world always have multiple types and naturally form the heterogeneous graph with rich structural and semantic information. As a result of this, it is beneficial to advance heterogeneous graph representation learning that can effectively promote the performance of complex network analysis. Existing survey papers of heterogeneous graph representation learning summarize all possible embedding techniques for graphs and make insufficient analysis for deep neural network models. To tackle this issue, in this paper, we systematically summarize and analyze existing heterogeneous graph neural networks (HGNNs) and categorize them based on their neural network architecture. Meanwhile, we collect commonly used heterogeneous graph datasets and summarize their statistical information. In addition, we compare the performances between HGNNs and shallow embedding models to show the powerful feature learning ability of HGNNs. Finally, we conclude the application scenarios of HGNNs and some possible future research directions. We hope that this paper can provide a useful framework for researchers who interested in HGNNs.
Similar content being viewed by others
References
Abu-El-Haija S, Perozzi B, Kapoor A et al (2019) Mixhop: higher-order graph convolutional architectures via sparsified neighborhood mixing. In: Proceedings of the 36th international conference on machine learning, pp 21–29
Altaf B, Akujuobi U, Yu L et al (2019) Dataset recommendation via variational graph autoencoder. In: Proceedings of the IEEE international conference on data mining, pp 11–20
Baytas IM, Xiao C, Wang F et al (2018) Heterogeneous hyper-network embedding. In: Proceedings of the IEEE international conference on data mining, pp 875–880
Bian R, Koh YS, Dobbie G et al (2019) Network embedding and change modeling in dynamic heterogeneous networks. In: Proceedings of the 42nd international ACM conference on research and development in information retrieval, pp 861–864
Bordes A, Chopra S, Weston J (2014) Question answering with subgraph embeddings. In: Proceedings of the conference on empirical methods in natural language processing, pp 615–620
Bruna J, Zaremba W, Szlam A et al (2014) Spectral networks and deep locally connected networks on graphs. In: Proceedings of the 2nd international conference on learning representations
Cai X, Han J, Yang L (2018) Generative adversarial network based heterogeneous bibliographic network representation for personalized citation recommendation. In: Proceedings of the 32nd AAAI conference on artificial intelligence, pp 5747–5754
Cen Y, Zou X, Zhang J et al (2019) Representation learning for attributed multiplex heterogeneous network. In: Proceedings of the 25th ACM international conference on knowledge discovery and data mining, pp 1358–1368
Chami I, Ying R, Re C et al (2019) Hyperbolic graph convolutional neural networks. In: Proceedings of the 33rd international conference on neural information processing systems, pp 4868–4879
Chang S, Han W, Tang J et al (2015) Heterogeneous network embedding via deep architectures. In: Proceedings of the 21st ACM international conference on knowledge discovery and data mining, pp 119–128
Chen H, Yin H, Wang W et al (2018) Pme: projected metric embedding on heterogeneous networks for link prediction. In: Proceedings of the 24th ACM international conference on knowledge discovery and data mining, pp 1177–1186
Cheng H, Yan X, Han J et al (2007) Discriminative frequent pattern analysis for effective classification. In: Proceedings of the IEEE 23rd international conference on data engineering, pp 716–725
Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th international conference on neural information processing systems, pp 3844–3852
Deshpande M, Kuramochi M, Wale N et al (2005) Frequent substructure-based approaches for classifying chemical compounds. IEEE Trans Knowl Data Eng 17(8):1036–1050
Dong Y, Chawla NV, Swami A (2017) metapath2vec: scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM international conference on knowledge discovery and data mining, pp 135–144
Fan S, Shi C, Wang X (2018) Abnormal event detection via heterogeneous information network embedding. In: Proceedings of the 27th ACM international conference on information and knowledge management, pp 1483–1486
Fan S, Zhu J, Han X et al (2019) Metapath-guided heterogeneous graph neural network for intent recommendation. In: Proceedings of the 25th ACM international conference on knowledge discovery and data mining, pp 2478–2486
Fang H, Wu F, Zhao Z et al (2016) Community-based question answering via heterogeneous social network learning. In: Proceedings of the 30th AAAI conference on artificial intelligence, pp 122–128
Fu Ty, Lee WC, Lei Z (2017) Hin2vec: explore meta-paths in heterogeneous information networks for representation learning. In: Proceedings of the 26th ACM on conference on information and knowledge management, pp 1797–1806
Fu X, Zhang J, Meng Z et al (2020) Magnn: metapath aggregated graph neural network for heterogeneous graph embedding. In: Proceedings of the 29th international conference on world wide web, pp 2331–2341
Goodfellow IJ, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial nets. In: Proceedings of the 27th international conference on neural information processing systems, pp 2672–2680
Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM international conference on knowledge discovery and data mining, pp 855–864
Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st international conference on neural information processing systems, pp 1025–1035
He Y, Song Y, Li J et al (2019) Hetespaceywalk: a heterogeneous spacey random walk for heterogeneous information network embedding. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 639–648
Hofmann T, Buhmann J (1994) Multidimensional scaling and data clustering. In: Proceedings of the 7th international conference on neural information processing systems, pp 459–466
Hong H, Guo H, Lin Y et al (2020) An attention-based graph neural network for heterogeneous structural learning. In: Proceedings of the 34th AAAI conference on artificial intelligence, pp 4132–4139
Horváth T, Gärtner T, Wrobel S (2004) Cyclic pattern kernels for predictive graph mining. In: Proceedings of the 10th ACM international conference on knowledge discovery and data mining, pp 158–167
Hossin M, Sulaiman MN (2015) A review on evaluation metrics for data classification evaluations. Int J Data Mining Knowl Manag Process 5(2):1
Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(1):193–218
Hu B, Fang Y, Shi C (2019a) Adversarial learning on heterogeneous information networks. In: Proceedings of the 25th ACM international conference on knowledge discovery and data mining, pp 120–129
Hu B, Zhang Z, Shi C et al (2019b) Cash-out user detection based on attributed heterogeneous information network with a hierarchical attention mechanism. In: Proceedings of the 33rd AAAI conference on artificial intelligence, pp 946–953
Hu Z, Dong Y, Wang K et al (2020) Heterogeneous graph transformer. In: Proceedings of the 29th international conference on world wide web, pp 2704–2710
Jiang R, Fu W, Wen L et al (2016) Dimensionality reduction on anchorgraph with an efficient locality preserving projection. Neurocomputing 187:109–118
Jiang X, Jia T, Fang Y et al (2021) Pre-training on large-scale heterogeneous graph. In: Proceedings of the 27th ACM conference on knowledge discovery and data mining, pp 756–766
Jin Z, Liu R, Li Q et al (2016) Predicting user’s multi-interests with network embedding in health-related topics. In: Proceedings of the IEEE international joint conference on neural networks, pp 2568–2575
Kashima H, Tsuda K, Inokuchi A (2003) Marginalized kernels between labeled graphs. In: Proceedings of the 20th international conference on machine learning, pp 321–328
Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114
Kipf TN, Welling M (2016) Variational graph auto-encoders. arXiv preprint arXiv:1611.07308
Lan T, Wu C, Yu C et al (2020) Adversarial network embedding on heterogeneous information networks. In: Journal of Physics: Conference Series, IOP Publishing, pp 012–018
Lee ML, Yang LH, Hsu W et al (2002) Xclust: clustering xml schemas for effective integration. In: Proceedings of the 11th international conference on information and knowledge management, pp 292–299
Lian W, Mamoulis N, Yiu SM et al (2004) An efficient and scalable algorithm for clustering xml documents by structure. IEEE Trans Knowl Data Eng 16(1):82–96
Li L, Gan Z, Cheng Y et al (2019) Relation-aware graph attention network for visual question answering. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10313–10322
Liu Z, Zheng VW, Zhao Z et al (2017) Semantic proximity search on heterogeneous graph by proximity embedding. In: Proceedings of the 31st AAAI conference on artificial intelligence, pp 154–160
Liu Z, Chen C, Yang X et al (2018) Heterogeneous graph neural networks for malicious account detection. In: Proceedings of the 27th ACM international conference on information and knowledge management, pp 2077–2085
Liu Q, Nickel M, Kiela D (2019) Hyperbolic graph neural networks. In: Proceedings of the 33rd international conference on neural information processing systems, pp 8230–8241
Madry A, Makelov A, Schmidt L et al (2018) Towards deep learning models resistant to adversarial attacks. In: Proceedings of the international conference on learning representations
Mikolov T, Chen K, Corrado G et al (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781
Milroy L, Llamas C (2013) Social networks. The handbook of language variation and change pp 407–427
Niepert M, Ahmed M, Kutzkov K (2016) Learning convolutional neural networks for graphs. In: Proceedings of the 33rd international conference on international conference on machine learning, pp 2014–2023
Pan S, Hu R, Long G et al (2018) Adversarially regularized graph autoencoder for graph embedding. In: Proceedings of the 27th international joint conference on artificial intelligence, pp 2609–2615
Park C, Kim D, Zhu Q et al (2019) Task-guided pair embedding in heterogeneous network. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 489–498
Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM international conference on knowledge discovery and data mining, pp 701–710
Pillai I, Fumera G, Roli F (2017) Designing multi-label classifiers that maximize f measures: state of the art. Pattern Recogn 61:394–404
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Ruan CY, Wang Y, Ma J et al (2019) Adversarial heterogeneous network embedding with metapath attention mechanism. J Comput Sci Technol 34(6):1217–1229
Shi C, Philip SY (2017) Heterogeneous information network analysis and applications. Springer, New York
Shi C, Li Y, Zhang J et al (2016) A survey of heterogeneous information network analysis. IEEE Trans Knowl Data Eng 29(1):17–37
Shiokawa H, Takahashi T, Kitagawa H (2018) Scalescan: scalable density-based graph clustering. In: Proceedings of the international conference on database and expert systems applications. Springer, pp 18–34
Strehl A, Ghosh J (2002) Cluster ensembles—a knowledge reuse framework for combining multiple partitions. J Mach Learn Res 3(Dec):583–617
Sun Y, Han J, Yan X et al (2011) Pathsim: meta path-based top-k similarity search in heterogeneous information networks. Proc VLDB Endow 4(11):992–1003
Tang J, Zhang J, Yao L et al (2008) Arnetminer: extraction and mining of academic social networks. In: Proceedings of the 14th ACM international conference on knowledge discovery and data mining, pp 990–998
Tang J, Qu M, Mei Q (2015a) Pte: predictive text embedding through large-scale heterogeneous text networks. In: Proceedings of the 21th ACM international conference on knowledge discovery and data mining, pp 1165–1174
Tang J, Qu M, Wang M et al (2015b) Line: large-scale information network embedding. In: Proceedings of the 24th international conference on world wide web, pp 1067–1077
Tian Z, Jia L, Dong H et al (2016) Analysis of urban road traffic network based on complex network. Procedia Eng 137:537–546
Tu K, Cui P, Wang X et al (2018) Structural deep embedding for hyper-networks. In: Proceedings of the 32nd AAAI conference on artificial intelligence, pp 426–433
van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9:2579–2605
Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. In: Proceedings of the 31st international conference on neural information processing systems, pp 6000–6010
Vázquez A, Flammini A, Maritan A et al (2003) Modeling of protein interaction networks. Complexus 1(1):38–44
Veličković P, Cucurull G, Casanova A et al (2018) Graph attention networks. In: Proceedings of the international conference on learning representations
Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM international conference on knowledge discovery and data mining, pp 1225–1234
Wang Q, Mao Z, Wang B et al (2017) Knowledge graph embedding: a survey of approaches and applications. IEEE Trans Knowl Data Eng 29(12):2724–2743
Wang H, Wang J, Wang J et al (2018a) Graphgan: graph representation learning with generative adversarial nets. In: Proceedings of the 32nd AAAI conference on artificial intelligence, pp 2508–2515
Wang H, Zhang F, Hou M et al (2018b) Shine: signed heterogeneous information network embedding for sentiment link prediction. In: Proceedings of the 11th ACM international conference on web search and data mining, pp 592–600
Wang X, Ji H, Shi C et al (2019) Heterogeneous graph attention network. In: Proceedings of the 28th international conference on world wide web, pp 2022–2032
Wang H, Wan R, Wen C et al (2020a) Author name disambiguation on heterogeneous information network with adversarial representation learning. In: Proceedings of the 34th AAAI conference on artificial intelligence, pp 238–245
Wang L, Song Y, Huang H et al (2020b) Modeling heterogeneous edges to represent networks with graph auto-encoder. In: Proceedings of the international conference on database systems for advanced applications, pp 498–515
Wang R, Zhu H, Wang L et al (2020c) User identity linkage across social networks by heterogeneous graph attention network modeling. Appl Sci 10(16):5478
Wang X, Liu N, Han H et al (2021) Self-supervised heterogeneous graph neural network with co-contrastive learning. In: Proceedings of the 27th conference on knowledge discovery and data mining, pp 1726–1736
Wang X, Bo D, Shi C et al (2022a) A survey on heterogeneous graph embedding: methods, techniques, applications and sources. IEEE Trans Big Data
Wang X, Lu Y, Shi C et al (2022b) Dynamic heterogeneous information network embedding with meta-path based proximity. IEEE Trans Knowl Data Eng 34(03):1117–1132
Welling M, Kipf TN (2016) Semi-supervised classification with graph convolutional networks. In: Proceedings of the international conference on learning representations
Wu J, He J, Xu J (2019) Net: degree-specific graph neural networks for node and graph classification. In: Proceedings of the 25th ACM international conference on knowledge discovery and data mining, pp 406–415
Wu Z, Pan S, Chen F et al (2020) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24
Xiang S, Nie F, Zhang C et al (2008) Nonlinear dimensionality reduction with local spline embedding. IEEE Trans Knowl Data Eng 21(9):1285–1298
Xu L, Wei X, Cao J et al (2017) Embedding of embedding (EOE) joint embedding for coupled heterogeneous networks. In: Proceedings of the 10th ACM international conference on web search and data mining, pp 741–749
Xu K, Hu W, Leskovec J et al (2018) How powerful are graph neural networks? In: Proceedings of the international conference on learning representations
Xu K, Chen H, Liu S et al (2019) Topology attack and defense for graph neural networks: an optimization perspective. In: Proceedings of the 28th international joint conference on artificial intelligence, pp 3961–3967
Xue H, Yang L, Jiang W et al (2020) Modeling dynamic heterogeneous network for link prediction using hierarchical attention with temporal RNN. In: Proceedings of the joint european conference on machine learning and knowledge discovery in databases, pp 282–298
Yang Y, Nie F, Xiang S et al (2010) Local and global regressive mapping for manifold learning with out-of-sample extrapolation. In: Proceedings of the AAAI conference on artificial intelligence, pp 649–654
Yang C, Xiao Y, Zhang Y et al (2020a) Heterogeneous network representation learning: a unified framework with survey and benchmark. IEEE Trans Knowl Data Eng
Yang L, Xiao Z, Jiang W et al (2020b) Dynamic heterogeneous graph embedding using hierarchical attentions. In: Advances in information retrieval: 42nd european conference on IR research, pp 425–432
Yang Y, Guan Z, Li J et al (2021) Interpretable and efficient heterogeneous graph convolutional network. IEEE Trans Knowl Data Eng
Yuan C, Li J, Zhou W et al (2020) Dyhgcn: a dynamic heterogeneous graph convolutional network to learn users’ dynamic preferences for information diffusion prediction. In: Proceedings of the joint european conference on machine learning and knowledge discovery in databases, pp 347–363
Yun S, Jeong M, Kim R et al (2019) Graph transformer networks. In: Proceedings of the 33rd international conference on neural information processing systems, pp 11983–11993
Zhang D, Yin J, Zhu X et al (2017a) User profile preserving social network embedding. In: Proceedings of the 26th international joint conference on artificial intelligence, pp 3378–3384
Zhang J, Xia C, Zhang C et al (2017b) Bl-mne: emerging heterogeneous social network embedding through broad learning with aligned autoencoder. In: Proceedings of the IEEE international conference on data mining, pp 605–614
Zhang C, Huang C, Yu L et al (2018a) Camel: content-aware and meta-path augmented metric learning for author identification. In: Proceedings of the 27th international conference on world wide web, pp 709–718
Zhang D, Yin J, Zhu X et al (2018b) Metagraph2vec: complex semantic path augmented heterogeneous network embedding. In: Proceedings of the pacific-asia conference on knowledge discovery and data mining, pp 196–208
Zhang C, Swami A, Chawla NV (2019a) Shne: representation learning for semantic-associated heterogeneous networks. In: Proceedings of the 12th ACM international conference on web search and data mining, pp 690–698
Zhang Y, Fan Y, Song W et al (2019b) Your style your identity: leveraging writing and photography styles for drug trafficker identification in darknet markets over attributed heterogeneous information network. In: Proceedings of the 28th international conference on world wide web, pp 3448–3454
Zhang Y, Fan Y, Ye Y et al (2019c) Key player identification in underground forums over attributed heterogeneous information network embedding framework. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 549–558
Zhang Y, Wang X, Shi C et al (2021) Hyperbolic graph attention network. IEEE Trans Big Data
Zhang Z, Cui P, Zhu W (2022) Deep learning on graphs: a survey. IEEE Trans Knowl Data Eng 34(1):249–270
Zhao A, Yu Y (2022) Context aware sentiment link prediction in heterogeneous social network. Cognitive Comput 14(1):300–309
Zhao Z, Yang Q, Cai D et al (2016) Expert finding for community-based question answering via ranking metric network learning. In: Proceedings of the 25th international joint conference on artificial intelligence, pp 3000–3006
Zhao J, Zhou Z, Guan Z et al (2019) Intentgc: a scalable graph convolution framework fusing heterogeneous information for recommendation. In: Proceedings of the 25th ACM international conference on knowledge discovery and data mining, pp 2347–2357
Zhao K, Bai T, Wu B et al (2020a) Deep adversarial completion for sparse heterogeneous information network embedding. In: Proceedings of the 29th international conference on world wide web, pp 508–518
Zhao R, Xiong X, Zu X et al (2020b) A hierarchical attention recommender system based on cross-domain social networks. Complexity 2020:1–13
Zhao J, Wang X, Shi C et al (2021a) Heterogeneous graph structure learning for graph neural networks. In: Proceedings of the 35th AAAI conference on artificial intelligence, pp 4697–4705
Zhao J, Wang X, Shi C et al (2021b) Network schema preserving heterogeneous information network embedding. In: Proceedings of the 29th international conference on international joint conferences on artificial intelligence, pp 1366–1372
Zhou S, Bu J, Wang X et al (2019) Hahe: hierarchical attentive heterogeneous information network embedding. arXiv preprint arXiv:1902.01475
Zhu S, Zhou C, Pan S et al (2019) Relation structure-aware heterogeneous graph neural network. In: Proceedings of the IEEE international conference on data mining, pp 1534–1539
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grants 71774159 and 62272066, State Key Laboratory of NBC Protection for Civilian under Grant SKLNBC2020-23, and the Fundamental Research Funds for the Central Universities of China under Grant 2015XKMS085.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bing, R., Yuan, G., Zhu, M. et al. Heterogeneous graph neural networks analysis: a survey of techniques, evaluations and applications. Artif Intell Rev 56, 8003–8042 (2023). https://doi.org/10.1007/s10462-022-10375-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-022-10375-2