Abstract
Omnipresent network/graph data generally have the characteristics of nonlinearity, sparseness, dynamicity and heterogeneity, which bring numerous challenges to network related analysis problem. Recently, influenced by the excellent ability of deep learning to learn representation from data, representation learning for network data has gradually become a new research hotspot. Network representation learning aims to learn a project from given network data in the original topological space to low-dimensional vector space, while encoding a variety of structural and semantic information. The vector representation obtained could effectively support extensive tasks such as node classification, node clustering, link prediction and graph classification. In this survey, we comprehensively present an overview of a large number of network representation learning algorithms from two clear points of view of homogeneous network and heterogeneous network. The corresponding algorithms are deeply analyzed. Extensive applications are introduced in an all-round way, and related experiments are conducted to validate the typical algorithms. Finally, we point out five future promising directions for next research in terms of theory and application.
Similar content being viewed by others
References
Lynch C (2008) Big data: how do your data grow? Nature 455(7209):28–29. https://doi.org/10.1038/455028a
Lazer D, Kennedy R, King G, Vespignani A (2014) The parable of Google Flu: traps in big data analysis. Science 343(6176):1203–1205. https://doi.org/10.1126/science.1248506
Howe D, Costanzo M, Fey P, Gojobori T, Hannick L, Hide W, Hill DP, Kania R, Schaeffer M, Pierre SS, Twigger S, White O, Rhee SY (2008) Big data: the future of biocuration. Nature 455(7209):47–50. https://doi.org/10.1038/455047a
Chang S, Han W, Tang J, Qi GJ, Aggarwal CC, Huang TS (2015) Heterogeneous network embedding via deep architectures. In: ACM SIGKDD international conference on knowledge discovery and data mining, pp 119–128. https://doi.org/10.1145/2783258.2783296
Zitnik M, Agrawal M, Leskovec J (2018) Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics 34:i457–i466. https://doi.org/10.1093/bioinformatics/bty294
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828. https://doi.org/10.1109/TPAMI.2013.50
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444. https://doi.org/10.1038/nature14539
Dahl GE, Ranzato MA, Mohamed AR, Hinton G (2010) Phone recognition with the mean-covariance restricted Boltzmann Machine. In: Proceedings of the 23rd international conference on neural information processing systems, pp 469–477
Mohamed A, Dahl GE, Hinton G (2012) Acoustic modeling using deep belief networks. IEEE Trans Audio Speech Lang Proc 20(1):14–22. https://doi.org/10.1109/tasl.2011.2109382
Hinton G, Deng L, Yu D, Dahl GE, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97. https://doi.org/10.1109/MSP.2012.2205597
Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 35(8):1915–1929. https://doi.org/10.1109/tpami.2012.231
Tompson J, Jain A, Lecun Y, Bregler C (2014) Joint training of a convolutional network and a graphical model for human pose estimation. In: Proceedings of the 27th international conference on neural information processing systems, pp 1799–1807
Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527
Bengio Y, Lamblin P, Popovici D, Larochelle H (2006) Greedy layer-wise training of deep networks. In: Proceedings of the 19th international conference on neural information processing systems, pp 153–160
Rifai S, Dauphin YN, Vincent P, Bengio Y, Muller X (2011) The manifold tangent classifier. In: Proceedings of the 24th international conference on neural information processing systems, pp 2294–2302
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Annual conference on neural information processing systems, pp 1106–1114
Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: Proceedings of the 2012 IEEE conference on computer vision and pattern recognition. IEEE Computer Society, 2354694, pp 3642–3649. https://doi.org/10.1109/cvpr.2012.6248110
Pan H, Wang B, Jiang H (2015) Deep learning for object saliency detection and image segmentation. arXiv
Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Annual conference on neural information processing systems, pp 3111–3119
Li Y, Xu L, Tian F, Jiang L, Zhong X, Chen E (2015) Word embedding revisited: a new representation learning and explicit matrix factorization perspective. In: International conference on artificial intelligence, pp 3650–3656
Le QV, Mikolov T (2014) Distributed representations of sentences and documents. In: Proceedings of the 31th international conference on machine learning, pp 1188–1196
Chien JT, Ku YC (2016) Bayesian recurrent neural network for language modeling. IEEE Trans Neural Netw Learn Syst 27(2):361–374. https://doi.org/10.1109/TNNLS.2015.2499302
Mikolov T, Corrado G, Chen K, Dean J, Mikolov T, Corrado G, Chen K, Dean J (2013) Efficient estimation of word representations in vector space. In: International conference on learning representations
Liben-Nowell D, Kleinberg J (2007) The link-prediction problem for social networks. J Assoc Inf Sci Technol 58(7):1019–1031. https://doi.org/10.1002/asi.v58:7
Ou M, Cui P, Pei J, Zhang Z, Zhu W (2016) Asymmetric transitivity preserving graph embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1105–1114. https://doi.org/10.1145/2939672.2939751
Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1225–1234. https://doi.org/10.1145/2939672.2939753
Grover A, Leskovec J (2016) node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 855–864. https://doi.org/10.1145/2939672.2939754
Tu C, Liu H, Liu Z, Sun M (2017) CANE: context-aware network embedding for relation modeling. In: Proceedings of the 55th annual meeting of the association for computational linguistics, pp 1722–1731. https://doi.org/10.18653/v1/p17-1158
Zhou C, Liu Y, Liu X, Liu Z, Gao J (2017) Scalable graph embedding for asymmetric proximity. In: Proceedings of the 31st AAAI conference on artificial intelligence, pp 2942–2948
Ribeiro LFR, Saverese PHP, Figueiredo DR (2017) struc2vec: learning node representations from structural identity. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 385–394. https://doi.org/10.1145/3097983.3098061
Yang Z, Cohen WW, Salakhutdinov R (2016) Revisiting semi-supervised learning with graph embeddings. In: Proceedings of the 33nd international conference on machine learning, pp 40–48
Perozzi B, Al-Rfou R, Skiena S (2014) DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 701–710. https://doi.org/10.1145/2623330.2623732
Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015) LINE: large-scale information network embedding. In: Proceedings of the 24th international conference on world wide web, pp 1067–1077. https://doi.org/10.1145/2736277.2741093
Cao S, Lu W, Xu Q (2015) GraRep: learning graph representations with global structural information. In: Proceedings of the 24th ACM international conference on information and knowledge management, pp 891–900. https://doi.org/10.1145/2806416.2806512
Yang C, Liu Z, Zhao D, Sun M, Chang EY (2015) Network representation learning with rich text information. In: Proceedings of the twenty-fourth international joint conference on artificial intelligence, pp 2111–2117
Tu C, Zhang W, Liu Z, Sun M (2016) Max-margin deepwalk: discriminative learning of network representation. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, pp 3889–3895
Li J, Zhu J, Zhang B (2016) Discriminative deep random walk for network classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics, pp 1004–1013
Pan S, Wu J, Zhu X, Zhang C, Wang Y (2016) Tri-party deep network representation. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, pp 1895–1901
Wang X, Cui P, Wang J, Pei J, Zhu W, Yang S (2017) Community preserving network embedding. In: Proceedings of the 31st AAAI conference on artificial intelligence, pp 203–209
Huang X, Li J, Hu X (2017) Label informed attributed network embedding. In: Proceedings of the tenth ACM international conference on web search and data mining, pp 731–739. https://doi.org/10.1145/3018661.3018667
Li B, Pi D (2019) Learning deep neural networks for node classification. Expert Syst Appl 137:324–334. https://doi.org/10.1016/j.eswa.2019.07.006
Chen J, Zhang Q, Huang X (2016) Incorporate group information to enhance network embedding. In: Proceedings of the 25th ACM international conference on information and knowledge management, pp 1901–1904. https://doi.org/10.1145/2983323.2983869
Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph representations. In: Proceedings of the 30th AAAI conference on artificial intelligence, pp 1145–1152
Cavallari S, Zheng VW, Cai H, Chang CC, Cambria E (2017) Learning community embedding with community detection and node embedding on graphs. In: ACM on conference on information and knowledge management, pp 377–386. https://doi.org/10.1145/3132847.3132925
Yang C, Liu M, Wang Z, Liu L, Han J (2017) Graph clustering with dynamic embedding. arXiv
Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2007) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51. https://doi.org/10.1109/TPAMI.2007.12
Tenenbaum JB, Silva VD, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323. https://doi.org/10.1126/science.290.5500.2319
Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326. https://doi.org/10.1126/science.290.5500.2323
Belkin M, Niyogi P (2003) Laplacian Eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396. https://doi.org/10.1162/089976603321780317
Vural E, Guillemot C (2016) Out-of-sample generalizations for supervised manifold learning for classification. IEEE Trans Image Process 25(3):1410–1424. https://doi.org/10.1109/TIP.2016.2520368
Hong D, Yokoya N, Zhu XX (2017) Learning a robust local manifold representation for hyperspectral dimensionality reduction. IEEE J Sel Top Appl Earth Obs Remote Sens 10(6):2960–2975. https://doi.org/10.1109/JSTARS.2017.2682189
Wang W, Yan Y, Nie F, Yan S, Sebe N (2018) Flexible manifold learning with optimal graph for image and video representation. IEEE Trans Image Process 27(6):2664–2675. https://doi.org/10.1109/TIP.2018.2810515
Cui P, Wang X, Pei J, Zhu W (2019) A survey on network embedding. IEEE Trans Knowl Data Eng 31(5):833–852. https://doi.org/10.1109/TKDE.2018.2849727
Moyano LG (2017) Learning network representations. Eur Phys J Spec Top 226(3):499–518. https://doi.org/10.1140/epjst/e2016-60266-2
Goyal P, Ferrara E (2018) Graph embedding techniques, applications, and performance: a survey. Knowl Based Syst 158:78–94. https://doi.org/10.1016/j.knosys.2018.03.022
Cai H, Zheng VW, Chang CC (2018) A comprehensive survey of graph embedding: problems, techniques and applications. IEEE Trans Knowl Data Eng 30:1616–1637. https://doi.org/10.1109/TKDE.2018.2807452
Hamilton WL, Ying R, Leskovec J (2017) Representation learning on graphs: methods and applications. IEEE Eng Bull 40:52–74
Keikha MM, Rahgozar M, Asadpour M (2018) Community aware random walk for network embedding. Knowl Based Syst 148:47–54. https://doi.org/10.1016/j.knosys.2018.02.028
Zhang Y, Shi Z, Feng D, Zhan X-X (2019) Degree-biased random walk for large-scale network embedding. Future Gener Comput Syst 100:198–209. https://doi.org/10.1016/j.future.2019.05.033
Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International conference on learning representations
Zhuang C, Ma Q (2018) Dual graph convolutional networks for graph-based semi-supervised classification. In: International world wide web conferences, pp 499–508. https://doi.org/10.1145/3178876.3186116
Gao H, Wang Z, Ji S (2018) Large-scale learnable graph convolutional networks. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1416–1424. https://doi.org/10.1145/3219819.3219947
Dai Q, Li Q, Tang J, Wang D (2018) Adversarial network embedding. In: Thirty-second AAAI conference on artificial intelligence
Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations
Wang H, Wang J, Wang J, Zhao M, Zhang W, Zhang F, Xie X, Guo M (2018) GraphGAN: graph representation learning with generative adversarial nets. In: Thirty-second AAAI conference on artificial intelligence
Pan S, Hu R, Long G, Jiang J, Yao L, Zhang C (2018) Adversarially regularized graph autoencoder for graph embedding. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, pp 2609–2615. https://doi.org/10.24963/ijcai.2018/362
Hu F, Zhu Y, Wu S, Wang L, Tan T (2019) Hierarchical graph convolutional networks for semi-supervised node classification. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence. https://doi.org/10.24963/ijcai.2019/630
Gao H, Pei J, Huang H (2019) ProGAN: network embedding via proximity generative adversarial network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1308–1316. https://doi.org/10.1145/3292500.3330866
Fu S, Liu W, Tao D, Zhou Y, Nie L (2020) HesGCN: Hessian graph convolutional networks for semi-supervised classification. Inf Sci 514:484–498. https://doi.org/10.1016/j.ins.2019.11.019
Debnath AK, Compadre RLLD, Debnath G, Shusterman AJ, Hansch C (1991) Structure–activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity. J Med Chem 34(2):786–797. https://doi.org/10.1021/jm00106a046
Toivonen H, Srinivasan A, King RD, Kramer S, Helma C (2003) Statistical evaluation of the predictive toxicology challenge 2000–2001. Bioinformatics 19(10):1183–1193. https://doi.org/10.1093/bioinformatics/btg130
Xinyi Z, Chen L (2019) Capsule graph neural network. In: International conference on learning representations
Borgwardt KM, Ong CS, Schönauer S, Vishwanathan SVN, Smola AJ, Kriegel H-P (2005) Protein function prediction via graph kernels. Bioinformatics 21(1):47–56. https://doi.org/10.1093/bioinformatics/bti1007
Wale N, Karypis G (2008) Comparison of descriptor spaces for chemical compound retrieval and classification. Knowl Inf Syst 14(3):347–375. https://doi.org/10.1109/ICDM.2006.39
Leskovec J, Kleinberg JM, Faloutsos C (2005) Graphs over time: densification laws, shrinking diameters and possible explanations. In: Proceedings of the eleventh ACM SIGKDD international conference on knowledge discovery and data mining, pp 177–187. https://doi.org/10.1145/1081870.1081893
Yanardag P, Vishwanathan SVN (2015) Deep graph kernels. In: ACM SIGKDD international conference on knowledge discovery and data mining, pp 1365–1374. https://doi.org/10.1145/2783258.2783417
Fouss F, Pirotte A, Renders JM, Saerens M (2007) Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Trans Knowl Data Eng 19(3):355–369. https://doi.org/10.1109/TKDE.2007.46
Mnih A, Hinton G (2009) A scalable hierarchical distributed language model. In: Proceedings of the 21st international conference on neural information processing systems, pp 1081–1088
Feng R, Yang Y, Hu W, Wu F, Zhuang Y (2018) Representation learning for scale-free networks. In: Thirty-second AAAI conference on artificial intelligence
Chen H, Perozzi B, Hu Y, Skiena S (2018) HARP: hierarchical representation learning for networks. In: Thirty-second AAAI conference on artificial intelligence
Feng N, Recht B, Re C, Wright SJ (2011) Hogwild: a lock-free approach to parallelizing stochastic gradient descent. In: Proceedings of the 24th international conference on neural information processing systems, pp 693–701
Bottou L (1998) Online algorithms and stochastic approximations. Cambridge University Press, Cambridge
Hochstenbach ME (2009) A Jacobi–Davidson type method for the generalized singular value problem. Linear Algebra Appl 431(3–4):471–487. https://doi.org/10.1016/j.laa.2009.03.003
Weimer M, Karatzoglou A, Smola A (2008) Improving maximum margin matrix factorization. Mach Learn 72(3):263–276. https://doi.org/10.1007/978-3-540-87479-9_12
Lin CJ (2007) Projected gradient methods for nonnegative matrix factorization. Neural Comput 19(10):2756–2779. https://doi.org/10.1162/neco.2007.19.10.2756
Guillamet D, Vitrià J, Schiele B (2003) Introducing a weighted non-negative matrix factorization for image classification. Pattern Recognit Lett 24(14):2447–2454. https://doi.org/10.1016/S0167-8655(03)00089-8
Liu W, Zheng N (2004) Non-negative matrix factorization based methods for object recognition. Pattern Recognit Lett 25(8):893–897. https://doi.org/10.1016/j.patrec.2004.02.002
Lee DD, Seung HS (2000) Algorithms for non-negative matrix factorization. In: Proceedings of the 13th international conference on neural information processing systems, pp 535–541
Klema VC, Laub AJ (1980) The singular value decomposition: its computation and some applications. IEEE Trans Autom Control 25(2):164–176. https://doi.org/10.1109/TAC.1980.1102314
Mnih V, Heess N, Graves A, Kavukcuoglu K (2014) Recurrent models of visual attention. In: Proceedings of the 27th international conference on neural information processing systems, pp 2204–2212
Donahue J, Krähenbühl P, Darrell T (2017) Adversarial feature learning. In: International conference on learning representations
Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: International conference on learning representations
Yu L, Zhang W, Wang J, Yu Y (2017) SeqGAN: sequence generative adversarial nets with policy gradient. In: Thirty-first AAAI conference on artificial intelligence
Li J, Monroe W, Shi T, Jean S, Ritter A, Jurafsky D (2017) Adversarial learning for neural dialogue generation. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 2157–2169
Wang J, Yu L, Zhang W, Gong Y, Xu Y, Wang B, Zhang P, Zhang D (2017) IRGAN: a minimax game for unifying generative and discriminative information retrieval models. In: International ACM SIGIR conference on research and development in information retrieval, pp 515–524. https://doi.org/10.1145/3077136.3080786
Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: International conference on neural information processing systems, pp 2672–2680
Bullinaria JA, Levy JP (2007) Extracting semantic representations from word co-occurrence statistics: a computational study. Behav Res Methods 39(3):510–526. https://doi.org/10.3758/BF03193020
Kipf TN, Welling M (2016) Variational graph auto-encoders. In: NIPS workshop on Bayesian deep learning
Wang C, Pan S, Hu R, Long G, Jiang J, Zhang C (2019) Attributed graph clustering: a deep attentional embedding approach. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, pp 3670–3676. https://doi.org/10.24963/ijcai.2019/509
Shi H, Fan H, Kwok JT (2020) Effective decoding in graph auto-encoder using triadic closure. In: Proceedings of the 34th AAAI conference on artificial intelligence
Tolstikhin I, Bousquet O, Gelly S, Schoelkopf B (2018) Wasserstein auto-encoders. In: International conference on learning representations
Donnat C, Zitnik M, Hallac D, Leskovec J (2018) Learning structural node embeddings via diffusion wavelets. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1320–1329. https://doi.org/10.1145/3219819.3220025
Shuman DI, Narang SK, Frossard P, Ortega A, Vandergheynst P (2013) The emerging field of signal processing on graphs: extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Process Mag 30(3):83–98. https://doi.org/10.1109/MSP.2012.2235192
Narayanan A, Chandramohan M, Chen L, Liu Y, Saminathan S (2016) subgraph2vec: learning distributed representations of rooted sub-graphs from large graphs. arXiv
Shervashidze N, Schweitzer P, Leeuwen EJ, Mehlhorn K, Borgwardt KM (2011) Weisfeiler–Lehman graph kernels. J Mach Learn Res 12(3):2539–2561
Vishwanathan SVN, Schraudolph NN, Kondor R, Borgwardt KM (2010) Graph kernels. J Mach Learn Res 11:1201–1242
Shervashidze N, Vishwanathan SVN, Petri TH, Mehlhorn K, Borgwardt KM (2009) Efficient graphlet kernels for large graph comparison. In: Proceedings of the twelfth international conference on artificial intelligence and statistics, pp 488–495
Shervashidze N, Borgwardt KM (2009) Fast subtree kernels on graphs. In: International conference on neural information processing systems, pp 1660–1668
Borgwardt KM, Kriegel HP (2005) Shortest-path kernels on graphs. In: IEEE International conference on data mining, pp 74–81. https://doi.org/10.1109/icdm.2005.132
Niepert M, Ahmed M, Kutzkov K (2016) Learning convolutional neural networks for graphs. In: Proceedings of the 33nd international conference on machine learning, pp 2014–2023
Luo Z, Liu L, Yin J, Li Y, Wu Z (2017) Deep learning of graphs with Ngram convolutional neural networks. IEEE Trans Knowl Data Eng 29(10):2125–2139. https://doi.org/10.1109/TKDE.2017.2720734
Mckay BD, Piperno A (2014) Practical graph isomorphism, II. J Symb Comput 60:94–112
Zhang M, Cui Z, Neumann M, Yixin C (2018) An end-to-end deep learning architecture for graph classification. In: Proceedings of the thirty-second AAAI conference on artificial intelligence, pp 4438–4445
Bruna J, Zaremba W, Szlam A, Lecun Y (2014) Spectral networks and locally connected networks on graphs. In: International conference on learning representations
Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: The 30th conference on neural information processing systems
Ying R, You J, Morris C, Ren X, Hamilton W, Leskovec J (2018) Hierarchical graph representation learning with differentiable pooling. In: Proceedings of the 32nd international conference on neural information processing systems, pp 4805–4815
Chen F, Pan S, Jiang J, Huo H, Long G (2019) DAGCN: dual attention graph convolutional networks. In: International joint conference on neural networks, pp 1–8. https://doi.org/10.1109/ijcnn.2019.8851698
Lee JB, Rossi RA, Kong X (2018) Graph classification using structural attention. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1666–1674. https://doi.org/10.1145/3219819.3219980
Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Proceedings of the 31st international conference on neural information processing systems, pp 3859–3869
Zhu D, Cui P, Zhang Z, Pei J, Zhu W (2018) High-order proximity preserved embedding for dynamic networks. IEEE Trans Knowl Data Eng 30:2134–2144. https://doi.org/10.1109/TKDE.2018.2822283
Zhou L, Yang Y, Ren X, Wu F, Zhuang Y (2018) Dynamic network embedding by modeling triadic closure process. In: Thirty-second AAAI conference on artificial intelligence, vol 5, pp 393–406
Manessi F, Rozza A, Manzo M (2020) Dynamic graph convolutional networks. Pattern Recognit 97:107000. https://doi.org/10.1016/j.patcog.2019.107000
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: International conference on neural information processing systems, pp 2787–2795
Bian J, Chang Y, Fu Y, Chen WY (2012) Learning to blend vitality rankings from heterogeneous social networks. Neurocomputing 97(1):390–397. https://doi.org/10.1016/j.neucom.2012.06.024
Shen Z, Ma KL, Eliassi-Rad T (2006) Visual analysis of large heterogeneous social networks by semantic and structural abstraction. IEEE Trans Vis Comput Graph 12(6):1427–1439. https://doi.org/10.1109/TVCG.2006.107
Sun Y, Norick B, Han J, Yan X, Yu PS, Yu X (2012) Integrating meta-path selection with user-guided object clustering in heterogeneous information networks. In: The 18th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1348–1356. https://doi.org/10.1145/2339530.2339738
Shi C, Li Y, Zhang J, Sun Y, Yu PS (2017) A survey of heterogeneous information network analysis. IEEE Trans Knowl Data Eng 29(1):17–37. https://doi.org/10.1109/TKDE.2016.2598561
Sun Y, Han J (2012) Mining heterogeneous information networks: a structural analysis approach. ACM SIGKDD Explor Newsl 14(2):20–28. https://doi.org/10.1145/2481244.2481248
Jacob Y, Denoyer L, Gallinari P (2014) Learning latent representations of nodes for classifying in heterogeneous social networks. In: Proceedings of the 7th ACM international conference on web search and data mining, New York, NY, USA. ACM, 2556225, pp 373–382. https://doi.org/10.1145/2556195.2556225
Nickel M, Tresp V, Kriegel HP (2011) A three-way model for collective learning on multi-relational data. In: International conference on machine learning, pp 809-816
Bordes A, Weston J, Collobert R, Bengio Y (2012) Learning structured embeddings of knowledge bases. In: AAAI Conference on artificial intelligence
Socher R, Chen D, Manning CD, Ng AY (2013) Reasoning with neural tensor networks for knowledge base completion. In: International conference on neural information processing systems, pp 926–934
Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Twenty-eighth AAAI conference on artificial intelligence, pp 1112–1119
Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Twenty-ninth AAAI conference on artificial intelligence, pp 2181–2187
Yang B, Yih W, He X, Gao J, Deng L (2015) Embedding entities and relations for learning and inference in knowledge bases. In: International conference on learning representations
Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Meeting of the association for computational linguistics and the international joint conference on natural language processing, pp 687–696
Jia Y, Wang Y, Lin H, Jin X, Cheng X (2016) Locally adaptive translation for knowledge graph embedding. In: Proceedings of the thirtieth conference on artificial intelligence, pp 992–998
Ji G, Liu K, He S, Zhao J (2016) Knowledge graph completion with adaptive sparse transfer matrix. In: Thirtieth AAAI conference on artificial intelligence, pp 985–991
Xie R, Liu Z, Sun M (2016) Representation learning of knowledge graphs with hierarchical types. In: International joint conference on artificial intelligence, pp 2965–2971
Nickel M, Rosasco L, Poggio T (2016) Holographic embeddings of knowledge graphs. In: Thirtieth AAAI conference on artificial intelligence, pp 1955–1961
Ruobing X, Zhiyuan L, Jia J, Huanbo L, Maosong S (2016) Representation learning of knowledge graphs with entity descriptions. In: Proceedings of the thirtieth AAAI conference on artificial intelligence
Xiao H, Huang M, Zhu X (2016) From one point to a manifold: knowledge graph embedding for precise link prediction. In: International joint conference on artificial intelligence
Welbl J, Riedel S, Bouchard G (2016) Complex embeddings for simple link prediction. In: International conference on machine learning, pp 2071–2080
Chen M, Tian Y, Yang M, Zaniolo C (2017) Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In: International joint conference on artificial intelligence
Liu H, Wu Y, Yang Y (2017) Analogical inference for multi-relational embeddings. In: International conference on machine learning, pp 2168–2178
Ebisu T, Ichise R (2018) TorusE: knowledge graph embedding on a Lie group. In: Thirty-second AAAI conference on artificial intelligence (AAAI-18)
Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2D knowledge graph embeddings. In: Thirty-second AAAI conference on artificial intelligence, pp 1811–1818
Guo S, Wang Q, Wang L, Wang B, Guo L (2018) Knowledge graph embedding with iterative guidance from soft rules. In: Thirty-second AAAI conference on artificial intelligence (AAAI-18)
Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives ZG (2007) DBpedia: a nucleus for a web of open data. In: International semantic web conference, pp 11–15
Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD conference, pp 1247–1250. https://doi.org/10.1145/1376616.1376746
Chen T, Sun Y (2017) Task-guided and path-augmented heterogeneous network embedding for author identification. In: Web search and data mining, pp 295–304. https://doi.org/10.1145/3018661.3018735
Shi Y, Gui H, Zhu Q, Kaplan L, Han J (2018) AspEm: embedding learning by aspects in heterogeneous information networks. In: Proceedings of the 2018 SIAM international conference on data mining. https://doi.org/10.1137/1.9781611975321.16
Fu TY, Lee WC, Lei Z (2017) HIN2Vec: explore meta-paths in heterogeneous information networks for representation learning. In: ACM International conference on information and knowledge management, pp 1797–1806. https://doi.org/10.1145/3132847.3132953
Harper FM, Konstan JA (2016) The MovieLens datasets: history and context. ACM Trans Interact Intell Syst 5(4):19. https://doi.org/10.1145/2827872
Tu K, Cui P, Wang X, Wang F, Zhu W (2018) Structural deep embedding for hyper-networks. In: Thirty-second AAAI conference on artificial intelligence (AAAI-2018)
Zheng VW, Cao B, Zheng Y, Xie X, Yang Q (2010) Collaborative filtering meets mobile recommendation: a user-centered approach. In: Twenty-fourth AAAI conference on artificial intelligence
Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference. Springer, pp 593–607. https://doi.org/10.1007/978-3-319-93417-4_38
Nathani D, Chauhan J, Sharma C, Kaul M (2019) Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th conference of the association for computational linguistics, pp 4710–4723. https://doi.org/10.18653/v1/p19-1466
Tang J, Qu M, Mei Q (2015) PTE: predictive text embedding through large-scale heterogeneous text networks. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1165–1174. https://doi.org/10.1145/2783258.2783307
Gui H, Liu J, Tao F, Jiang M, Norick B, Han J (2016) Large-scale embedding learning in heterogeneous event data. In: IEEE International conference on data mining, pp 907–912. https://doi.org/10.1109/icdm.2016.0111
Gui H, Liu J, Tao F, Jiang M, Norick B, Kaplan L, Han J (2017) Embedding learning with events in heterogeneous information networks. IEEE Trans Knowl Data Eng 29(11):2428–2441. https://doi.org/10.1109/TKDE.2017.2733530
Berge C (1989) Hypergraphs: combinatorics of finite sets. Elsevier, Amsterdam
Silva JG, Willett R (2009) Hypergraph-based anomaly detection of high-dimensional co-occurrences. IEEE Trans Pattern Anal Mach Intell 31(3):563–569. https://doi.org/10.1109/TPAMI.2008.232
Zhang C, Song D, Huang C, Swami A, Chawla NV (2019) Heterogeneous graph neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery and data mining, pp 793–803. https://doi.org/10.1145/3292500.3330961
Wang X, Ji H, Shi C, Wang B, Ye Y, Cui P, Yu PS (2019) Heterogeneous graph attention network. In: The world wide web conference, pp 2022–2032. https://doi.org/10.1145/3308558.3313562
Zhu J, Zhang J, He L, Wu Q, Zhou B, Zhang C, Yu PS (2017) Broad learning based multi-source collaborative recommendation. In: ACM International conference on information and knowledge management, pp 1409–1418. https://doi.org/10.1145/3132847.3132976
Zhang J, Xia C, Zhang C, Cui L, Fu Y, Yu PS (2017) BL-MNE: emerging heterogeneous social network embedding through broad learning with aligned autoencoder. In: International conference on data mining, pp 605–614. https://doi.org/10.1109/icdm.2017.70
Sun Y, Han J (2012) Mining heterogeneous information networks: principles and methodologies. Morgan & Claypool Publishers, San Rafael
Sun Y, Han J, Yan X, Yu PS, Wu T (2011) PathSim: meta path-based top-k similarity search in heterogeneous information networks. In: Proceedings of the VLDB endowment, vol 11, pp 992–1003
Dong Y, Chawla NV, Swami A (2017) metapath2vec: scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, Halifax, NS, Canada. ACM, 3098036, pp 135–144. https://doi.org/10.1145/3097983.3098036
Zhang D, Yin J, Zhu X, Zhang C (2018) MetaGraph2Vec: complex semantic path augmented heterogeneous network embedding. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 196–208. https://doi.org/10.1007/978-3-319-93037-4_16
Shi C, Hu B, Zhao WX, Philip SY (2019) Heterogeneous information network embedding for recommendation. IEEE Trans Knowl Data Eng 31(2):357–370. https://doi.org/10.1109/TKDE.2018.2833443
Zhang H, Wang S, Xu X, Chow TW, Wu QJ (2018) Tree2Vector: learning a vectorial representation for tree-structured data. IEEE Trans Neural Netw Learn Syst 99:1–15. https://doi.org/10.1109/TNNLS.2018.2797060
Lu L, Zhang H (2015) A tree-structured representation for book author and its recommendation using multilayer SOM. In: 2015 International joint conference on neural networks (IJCNN). IEEE, pp 1–8. https://doi.org/10.1109/ijcnn.2015.7280530
Kazienko P, Kajdanowicz T (2012) Label-dependent node classification in the network. Neurocomputing 75(1):199–209. https://doi.org/10.1016/j.neucom.2011.04.047
Sun Y, Yuan Y, Wang G (2015) An on-line sequential learning method in social networks for node classification. Neurocomputing 149:207–214. https://doi.org/10.1016/j.neucom.2014.04.074
Bojchevski A, Günnemann S (2018) Deep Gaussian embedding of graphs: unsupervised inductive learning via ranking. In: International conference on learning representations
Fan R-E, Chang K-W, Hsieh C-J, Wang X-R, Lin C-J (2008) LIBLINEAR: a library for large linear classification. J Mach Learn Res 9:1871–1874
Peng W, Baowen X, Yurong W, Xiaoyu Z (2015) Link prediction in social networks: the state-of-the-art. Sci China Inf Sci 58(1):1–38. https://doi.org/10.1007/s11432-014-5237-y
Cannistraci CV, Alanis-Lobato G, Ravasi T (2013) Minimum curvilinearity to enhance topological prediction of protein interactions by network embedding. Bioinformatics 29(13):199–209. https://doi.org/10.1093/bioinformatics/btt208
Ma J, Cui P, Zhu W (2018) DepthLGP: learning embeddings of out-of-sample nodes in dynamic networks. In: Thirty-second AAAI conference on artificial intelligence
Fout A, Byrd J, Shariat B, Benhur A (2017) Protein interface prediction using graph convolutional networks. In: Proceedings of the 31st international conference on neural information processing systems, pp 6533–6542
Godfrey P, Gryz J, Lasek P (2016) Interactive visualization of large datasets. IEEE Trans Knowl Data Eng 28(8):2142–2157. https://doi.org/10.1109/TKDE.2016.2557324
Maaten LVD, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(2605):2579–2605
Tang J, Liu J, Zhang M, Mei Q (2016) Visualizing large-scale and high-dimensional Data. In: International conference on world wide web, vol 7, pp 287–297. https://doi.org/10.1145/2872427.2883041
Bobadilla J, Ortega F, Hernando A (2013) Recommender systems survey. Knowl Based Syst 46(1):109–132. https://doi.org/10.1016/j.knosys.2013.03.012
Chumki B, Haym H, Cohen W (1998) Recommendation as classification: using social and content-based information in recommendation. In: Proceedings of the fifteenth national conference on artificial intelligence and tenth innovative applications of artificial intelligence conference, pp 714–720
Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: International conference on world wide web, pp 285–295
Kitsak M, Gallos LK, Havlin S, Liljeros F, Muchnik L, Stanley HE, Makse HA (2010) Identification of influential spreaders in complex networks. Nat Phys 6(11):888–893. https://doi.org/10.1038/nphys1746
Schietgat L, Ramon J, Bruynooghe M (2013) A polynomial-time maximum common subgraph algorithm for outerplanar graphs and its application to chemoinformatics. Ann Math Artif Intell 69(4):343–376. https://doi.org/10.1007/s10472-013-9335-0
Deshpande M, Kuramochi M, Karypis G (2005) Frequent sub-structure-based approaches for classifying chemical compounds. In: IEEE International conference on data mining, pp 35–42. https://doi.org/10.1109/tkde.2005.127
Cheng H, Lo D, Zhou Y, Wang X, Yan X (2009) Identifying bug signatures using discriminative graph mining. In: Proceedings of the eighteenth international symposium on software testing and analysis, pp 141–152. https://doi.org/10.1145/1572272.1572290
Zhu X, Zhang C, Pan S, Yu PS (2013) Graph stream classification using labeled and unlabeled graphs. In: IEEE International conference on data engineering, pp 398–409. https://doi.org/10.1109/icde.2013.6544842
Dai H, Dai B, Song L (2016) Discriminative embeddings of latent variable models for structured data. In: International conference on machine learning, pp 2702–2711
You J, Liu B, Ying R, Pande V, Leskovec J (2018) Graph convolutional policy network for goal-directed molecular graph generation. In: Thirty-second conference on neural information processing systems
Wang Q, Mao Z, Wang B, Guo L (2017) Knowledge graph embedding: a survey of approaches and applications. IEEE Trans Knowl Data Eng 29(12):2724–2743. https://doi.org/10.1109/TKDE.2017.2754499
Dai H, Khalil EB, Zhang Y, Dilkina B, Song L (2017) Learning combinatorial optimization algorithms over graphs. In: Proceedings of the 31st international conference on neural information processing systems, pp 6351–6361
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
Sun M, Zhao S, Gilvary C, Elemento O, Zhou J, Wang F (2019) Graph convolutional networks for computational drug development and discovery. Brief Bioinform. https://doi.org/10.1093/bib/bbz042
Guo K, Hu Y, Qian Z, Liu H, Zhang K, Sun Y, Gao J, Yin B (2020) Optimized graph convolution recurrent neural network for traffic prediction. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2019.2963722
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. In: International conference on learning representations
Ahmed A, Shervashidze N, Narayanamurthy S, Josifovski V, Smola AJ (2013) Distributed large-scale natural graph factorization. In: International conference on world wide web, pp 37–48. https://doi.org/10.1145/2488388.2488393
Neumann M, Garnett R, Bauckhage C, Kersting K (2016) Propagation kernels: efficient graph kernels from propagated information. Mach Learn 102(2):209–245. https://doi.org/10.1007/s10994-015-5517-9
Simonovsky M, Komodakis N (2017) Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3693–3702. https://doi.org/10.1109/cvpr.2017.11
Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12(7):257–269
Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th international conference on neural information processing systems, pp 2787–2795
Thomas JM, Muscoloni A, Ciucci S, Bianconi G, Cannistraci CV (2017) Machine learning meets complex networks via coalescent embedding in the hyperbolic space. Nat Commun 8(1):1615. https://doi.org/10.1038/s41467-017-01825-5
Chamberlain BP, Clough J, Deisenroth MP (2017) Neural embeddings of graphs in hyperbolic space. arXiv
Dai H, Li H, Tian T, Huang X, Wang L, Zhu J, Song L (2018) Adversarial attack on graph structured data. In: International conference on machine learning, pp 1123–1132. https://doi.org/10.1145/3219819.3220078
Zügner D, Akbarnejad A, Günnemann S (2018) Adversarial attacks on neural networks for graph data. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining, London, UK. ACM, pp 2847–2856
Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. In: International conference on learning representations
Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, Tacchetti A, Raposo D, Santoro A, Faulkner R, Gulcehre C, Song F, Ballard A, Gilmer J, Dahl G, Vaswani A, Allen K, Nash C, Langston V, Dyer C, Heess N, Wierstra D, Kohli P, Botvinick M, Vinyals O, Li Y, Pascanu R (2018) Relational inductive biases, deep learning, and graph networks. arXiv
Bareinboim E, Pearl J (2016) Causal inference and the data-fusion problem. Proc Natl Acad Sci 113(27):7345–7352. https://doi.org/10.1073/pnas.1510507113
Cao B, He L, Xiaokai, Xing M, Yu PS, Klumpp H, Leow AD (2017) t-BNE: tensor-based brain network embedding. In: SIAM International conference on data mining, pp 189–197. https://doi.org/10.1137/1.9781611974973.22
Ma G, Lu CT, He L, Yu PS, Ragin AB (2017) Multi-view graph embedding with hub detection for brain network analysis. In: IEEE International conference on data mining, pp 967–972. https://doi.org/10.1109/icdm.2017.123
Camacho DM, Collins KM, Powers RK, Costello JC, Collins JJ (2018) Next-generation machine learning for biological networks. Cell 173(7):1581–1592. https://doi.org/10.1016/j.cell.2018.05.015
Acknowledgements
This research is supported by the National Natural Science Foundation of China (Grant Number: U1433116), and the Fundamental Research Funds for the Central Universities (Grant Number: NP2017208). Thanks are due to all developers and researchers providing open-source platform. Thanks are due to all the anonymous reviewers.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We declare that we have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, B., Pi, D. Network representation learning: a systematic literature review. Neural Comput & Applic 32, 16647–16679 (2020). https://doi.org/10.1007/s00521-020-04908-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-04908-5