Skip to main content
SpringerLink
Log in

Learning network representations

  • Review
  • A review with applications to complex networks
  • Published:
The European Physical Journal Special Topics Aims and scope Submit manuscript

Abstract

In this review I present several representation learning methods, and discuss the latest advancements with emphasis in applications to network science. Representation learning is a set of techniques that has the goal of efficiently mapping data structures into convenient latent spaces. Either for dimensionality reduction or for gaining semantic content, this type of feature embeddings has demonstrated to be useful, for example, for node classification or link prediction tasks, among many other relevant applications to networks. I provide a description of the state-of-the-art of network representation learning as well as a detailed account of the connections with other fields of study such as continuous word embeddings and deep learning architectures. Finally, I provide a broad view of several applications of these techniques to networks in various domains.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. G. Alanis-Lobato, Mining protein interactomes to improve their reliability and support the advancement of network medicine, Front. Genet. 6, 296 (2015)

    Article  Google Scholar 

  2. G. Alanis-Lobato, C.V. Cannistraci, A. Eriksson, A. Manica, T. Ravasi, Highlighting nonlinear patterns in population genetics datasets, Sci. Rep. 5, 8140 (2015)

    Article  ADS  Google Scholar 

  3. G. Alanis-Lobato, C.V. Cannistraci, T. Ravasi, Exploitation of genetic interaction network topology for the prediction of epistatic behavior, Genomics 102, 202 (2013)

    Article  Google Scholar 

  4. G. Alanis-Lobato, P. Mier, M.A. Andrade-Navarro, Efficient embedding of complex networks to hyperbolic space via their Laplacian, Sci. Rep. 6, 30108 (2016)

    Article  ADS  Google Scholar 

  5. T. Aste, T. Di Matteo, S.T. Hyde, Complex networks on hyperbolic surfaces, Physica A 346, 20 (2005)

    Article  ADS  Google Scholar 

  6. T. Aste, R. Gramatica, T.D. Matteo, Exploring complex networks via topological embedding on surfaces, Phys. Rev. E 86, 036109 (2012)

    Article  ADS  Google Scholar 

  7. M. Baroni, G. Dinu, G. Kruszewski, Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors, in Proceedings of Association for Computational Linguistics (ACL, 2014), p. 238

  8. M. Barthélemy, Spatial networks, Phys. Rep. 499, 1 (2011)

    Article  ADS  MathSciNet  Google Scholar 

  9. M. Belkin, P. Niyogi, in Laplacian eigenmaps and spectral techniques for embedding and clustering (NIPS, 2014), Vol 14, pp. 585–591

  10. M. Belkin, P. Niyogi, V. Sindhwani, Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, J. Mach. Learn. Res. 7, 2399 (2006)

    MathSciNet  MATH  Google Scholar 

  11. Y. Bengio, Learning deep architectures for AI, Found. Trends® Mach. Learn. 2, 1 (2009)

    Article  MATH  Google Scholar 

  12. Y. Bengio, A. Courville, P. Vincent, Representation learning: A review and new perspectives, IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798 (2013)

    Article  Google Scholar 

  13. D.M. Blei, A.Y. Ng, M.I. Jordan, Latent dirichlet allocation, J. Mach. Learn. Res. 3, 993 (2003)

    MATH  Google Scholar 

  14. V.D. Blondel, J.-L. Guillaume, R. Lambiotte, E. Lefebvre, Fast unfolding of communities in large networks, J. Stat. Mech.: Theor. Exp. 2008, P10008 (2008)

    Article  Google Scholar 

  15. M. Boguñá, D. Krioukov, Navigating ultrasmall worlds in ultrashort time, Phys. Rev. Lett. 102, 058701 (2009)

    Article  ADS  Google Scholar 

  16. M. Boguñá, D. Krioukov, K.C. Claffy, Navigability of complex networks, Nat. Phys. 5, 74 (2009)

    Article  Google Scholar 

  17. M. Boguñá, F. Papadopoulos, D. Krioukov, Sustaining the internet with hyperbolic mapping, Nat. commun. 1, 62 (2010)

    Article  ADS  Google Scholar 

  18. S. Bourigault, C. Lagnier, S. Lamprier, L. Denoyer, P. Gallinari, Learning social network embeddings for predicting information diffusion, in Proceedings of the 7th ACM international conference on Web search and data mining (ACM, 2014), p. 393

  19. C.V. Cannistraci, G. Alanis-Lobato, T. Ravasi, Minimum curvilinearity to enhance topological prediction of protein interactions by network embedding, Bioinformatics, 29, i199 (2013)

    Article  Google Scholar 

  20. S. Cao, W. Lu, Q. Xu, GraRep: Learning graph representations with global structural information, in Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (ACM, 2015), p. 891

  21. S. Cao, W. Lu, Q. Xu, Deep neural networks for learning graph representations, in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16) (AAAI Press, 2016)

  22. P.R. Cavalin, L.G. Moyano, P.P. Miranda, A multiple classifier system for classifying life events on social media, in 2015 IEEE International Conference on Data Mining Workshop (ICDMW) (IEEE, 2015), pp. 1332–1335

  23. S. Chang, W. Han, J. Tang, G.-J. Qi, C.C. Aggarwal, T.S. Huang, Heterogeneous network embedding via deep architectures, in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2015), p. 119

  24. W. Chen, W. Fang, G. Hu, M.W. Mahoney, On the hyperbolicity of small-world and treelike random graphs, Int. Math. 9, 434 (2013)

    MathSciNet  MATH  Google Scholar 

  25. R.F. Cohen, P. Eades, T. Lin, F. Ruskey, Three-dimensional graph drawing, Algorithmica 17, 199 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  26. A. Dallmann, T. Niebler, F. Lemmerich, A. Hotho, Extracting semantics from random walks on wikipedia: Comparing learning and counting methods, in Tenth International AAAI Conference on Web and Social Media (2016)

  27. L. Daqing, K. Kosmidis, A. Bunde, S. Havlin, Dimension of spatially embedded networks, Nat. Phys. 7, 481 (2011)

    Article  Google Scholar 

  28. S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, R. Harshman, Indexing by latent semantic analysis, J. Amer. Soc. Inf. Sci. 41, 391 (1990)

    Article  Google Scholar 

  29. R.O. Duda, P.E. Hart, D.G. Stork, Pattern classification (John Wiley & Sons, 2012)

  30. J. Ganesh, S. Ganguly, M. Gupta, V. Varma, V. Pudi, Author2vec: Learning author representations by combining content and link information, in Proceedings of the 25th International Conference Companion on World Wide Web, International World Wide Web Conferences Steering Committee (2016), p. 49

  31. L. Getoor, B. Taskar, Introduction to statistical relational learning (MIT Press, Cambridge, 2007)

  32. A. Grover, J. Leskovec, node2vec: Scalable feature learning for networks, in Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2016)

  33. M.S. Handcock, A.E. Raftery, J.M. Tantrum, Model-based clustering for social networks, J. R. Stat. Soc.: Ser. Stat. Soc. 170, 301 (2007)

    Article  MathSciNet  Google Scholar 

  34. X. He, P. Niyogi, Locality preserving projections, in Neural Information Processing Systems (MIT, 2004), Vol 16, p. 153

  35. L. Heck, Deep learning of knowledge graph embeddings for semantic parsing of twitter dialogs, in The 2nd IEEE Global Conference on Signal and Information Processing (IEEE, 2014)

  36. G.E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets, Neur. Comput. 18, 1527 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  37. L. Huang, J. May, X. Pan, H. Ji, Building a fine-grained entity typing system overnight for a new x (x= language, domain, genre) arXiv:1603.03112 (2016)

  38. P.-S. Huang, X. He, J. Gao, L. Deng, A. Acero, L. Heck, Learning deep structured semantic models for web search using clickthrough data, in Proceedings of the 22Nd ACM International Conference on Information & Knowledge Management, CIKM ’13, (New York, USA, 2013, ACM), p. 2333

  39. Y. Jacob, L. Denoyer, P. Gallinari, Classification dans les graphes hétérogènes basée sur une représentation latente des nœuds, in CORIA 2013 (2013), pp. 85–100

  40. Y. Jacob, L. Denoyer, P. Gallinari, Learning latent representations of nodes for classifying in heterogeneous social networks, in Proceedings of the 7th ACM international conference on Web search and data mining (ACM, 2014), p. 373

  41. R. Kiros, R. Zemel, R.R. Salakhutdinov, A multiplicative model for learning distributed text-based attribute representations, in Advances in Neural Information Processing Systems (2014), pp. 2348–2356

  42. J.M. Kleinberg, Navigation in a small world, Nature 406, 845 (2000)

    Article  ADS  Google Scholar 

  43. R. Kleinberg, Geographic routing using hyperbolic space, in IEEE INFOCOM 2007-26th IEEE International Conference on Computer Communications (IEEE, 2007), pp. 1902–1909

  44. D. Krioukov, F. Papadopoulos, M. Boguñá, A. Vahdat, Greedy forwarding in scale-free networks embedded in hyperbolic metric spaces, ACM SIGMETRICS Perform. Eval. Rev. 37, 15 (2009)

    Article  Google Scholar 

  45. D. Krioukov, F. Papadopoulos, M. Kitsak, A. Vahdat, M. Boguná, Hyperbolic geometry of complex networks, Phys. Rev. E 82, 036106 (2010)

    Article  ADS  MathSciNet  Google Scholar 

  46. D. Krioukov, F. Papadopoulos, A. Vahdat, M. Boguñá, Curvature and temperature of complex networks, Phys. Rev. E 80, 035101 (2009)

    Article  ADS  Google Scholar 

  47. C. Lagnier, S. Bourigault, S. Lamprier, L. Denoyer, P. Gallinari, Learning information spread in content networks, arXiv:1312.6169 (2014)

  48. Y.-Y. Lai, C. Li, D. Goldwasser, J. Neville, Better together: Combining language and social interactions into a shared representation, in Proceedings of TextGraphs-10: the Workshop on Graph-based Methods for Natural Language Processing (San Diego, CA, USA, 2016. Association for Computational Linguistics), p. 29

  49. Q. Le, T. Mikolov, Distributed representations of sentences and documents, in Proceedings of The 31st International Conference on Machine Learning (ICML, 2014), Vol 14, p. 1188

  50. Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521, 436 (2015)

    Article  ADS  Google Scholar 

  51. O. Levy, Y. Goldberg, Neural word embedding as implicit matrix factorization, in Advances in Meural Information Processing Systems (2014), pp. 2177–2185

  52. K. Li, J. Gao, S. Guo, N. Du, X. Li, A. Zhang, Lrbm: A restricted boltzmann machine based approach for representation learning on linked data, in 2014 IEEE International Conference on Data Mining (IEEE, 2014), pp. 300–309

  53. Y. Li, L. Xu, F. Tian, L. Jiang, X. Zhong, E. Chen, Word embedding revisited: A new representation learning and explicit matrix factorization perspective, in Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI, 2015), p. 25

  54. D. Liben-Nowell, J. Kleinberg, The link-prediction problem for social networks, J. Amer. Soc. Inf. Sci. Tech. 58, 1019 (2007)

    Article  Google Scholar 

  55. F. Liu, B. Liu, C. Sun, M. Liu, X. Wang, Deep learning approaches for link prediction in social network services, in International Conference on Neural Information Processing (Springer, 2013), pp. 425–432

  56. F. Liu, B. Liu, C. Sun, M. Liu, X. Wang, Deep belief network-based approaches for link prediction in signed social networks, Entropy 17, 2140 (2015)

    Article  ADS  Google Scholar 

  57. Y. Long, Characterizing video diffusion patterns in online social networks, HKU Theses Online (HKUTO), 2015

  58. K. Lu, Z. Ding, S. Ge, Sparse-representation-based graph embedding for traffic sign recognition, IEEE Trans. Intell. Trans. Syst. 13, 1515 (2012)

    Article  Google Scholar 

  59. Y. Luo, Q. Wang, B. Wang, L. Guo, Context-dependent knowledge graph embedding, in EMNLP, editor, Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (ACL, 2015), p. 1656

  60. S.A. Macskassy, F. Provost, Classification in networked data: A toolkit and a univariate case study, J. Mach. Learn. Res. 8, 935 (2007)

    Google Scholar 

  61. T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space, in Workshop paper at International Conference on Learning Representations (ICLR, 2013)

  62. T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, J. Dean, Distributed representations of words and phrases and their compositionality, in Advances in neural information processing systems, (2013), pp. 3111–3119

  63. L.G. Moyano, J.P. Cárdenas, J. Salcedo, M.L. Mouronte, R.M. Benito, Information transfer dynamics in fixed-pathways networks, Chaos 21, 013126 (2011)

    Article  ADS  Google Scholar 

  64. S. Nandanwar, M. Murty, Structural neighborhood based classification of nodes in a network, in Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2016)

  65. O. Narayan, I. Saniee, Large-scale curvature of networks, Phys. Rev. E. 84, 066108 (2011)

    Article  ADS  Google Scholar 

  66. L. Niu, X.-Y. Dai, S. Huang, J. Chen, A unified framework for jointly learning distributed representations of word and attributes, in Proceedings of The 7th Asian Conference on Machine Learning (2015), p. 143

  67. D. Nozza, D. Maccagnola, V. Guigue, E. Messina, P. Gallinari, A latent representation model for sentiment analysis in heterogeneous social networks, in International Conference on Software Engineering and Formal Methods (Springer, 2014), pp. 201–213

  68. F. Papadopoulos, M. Kitsak, M.Á. Serrano, M. Boguná, D. Krioukov, Popularity versus similarity in growing networks, Nature 489, 537 (2012)

    Article  ADS  Google Scholar 

  69. F. Papadopoulos, D. Krioukov, M. Boguna, A. Vahdat, Greedy forwarding in dynamic scale-free networks embedded in hyperbolic metric spaces, in INFOCOM, 2010 Proceedings IEEE, (IEEE, 2010), p. 1

  70. C.D. Manning, J. Pennington, R. Socher, Glove: Global vectors for word representation, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language (2014)

  71. B. Perozzi, R. Al-Rfou, S. Skiena, Deepwalk: Online learning of social representations, in Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining (ACM, 2014), p. 701

  72. B. Perozzi, V. Kulkarni, S. Skiena, Walklets: Multiscale graph embeddings for interpretable network classification, arXiv:1605.02115 (2016)

  73. X. Ren, W. He, M. Qu, C.R. Voss, H. Ji, J. Han, Label noise reduction in entity typing by heterogeneous partial-label embedding, in Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2016)

  74. S.T. Roweis, L.K. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science 290, 2323 (2000)

    Article  ADS  Google Scholar 

  75. P. Sanguansat, Paragraph2vec-based sentiment analysis on social media for business in thailand, in 2016 8th International Conference on Knowledge and Smart Technology (KST) (IEEE, 2016), pp. 175–178

  76. P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, T. Eliassi-Rad, Collective classification in network data, AI magazine 29, 93 (2008)

    Google Scholar 

  77. M.A. Serrano, D. Krioukov, M. Boguná, Self-similarity of complex networks and hidden metric spaces, Phys. Rev. Lett. 100, 078701 (2008)

    Article  ADS  Google Scholar 

  78. Y. Shavitt, T. Tankel, Big-bang simulation for embedding network distances in euclidean space, IEEE/ACM Trans. Networking (TON) 12, 993 (2004)

    Article  Google Scholar 

  79. Y. Shavitt, T. Tankel, On the curvature of the internet and its usage for overlay construction and distance estimation, in INFOCOM 2004. Twenty-third AnnualJoint Conference of the IEEE Computer and Communications Societies (IEEE, 2004), Vol 1

  80. Y. Shavitt, T. Tankel, Hyperbolic embedding of internet graph for distance estimation and overlay construction, IEEE/ACM Trans. Networking (TON) 16, 25 (2008)

    Article  Google Scholar 

  81. T.A.B. Snijders, K. Nowicki, Estimation and prediction for stochastic blockmodels for graphs with latent block structure, J. Classification 14, 75 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  82. J. Tang, J. Liu, M. Zhang, Q. Mei, Visualizing large-scale and high-dimensional data, in Proceedings of the 25th International Conference on World Wide Web (International World Wide Web Conferences Steering Committee, 2016), p. 287

  83. J. Tang, M. Qu, Q. Mei, Pte: Predictive text embedding through large-scale heterogeneous text networks, in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2015), p. 1165

  84. J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, Q. Mei, Line: Large-scale information network embedding, in Proceedings of the 24th International Conference on World Wide Web (ACM, 2015), p. 1067

  85. L. Tang, H. Liu, Relational learning via latent social dimensions, in Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (ACM, 2009), p. 817

  86. L. Tang, H. Liu, Leveraging social media networks for classification, Data Mining Knowl. Discovery 23, 447 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  87. J.B. Tenenbaum, V. De Silva, J.C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science 290, 2319 (2000)

    Article  ADS  Google Scholar 

  88. F. Tian, B. Gao, Q. Cui, E. Chen, T.-Y. Liu, Learning deep representations for graph clustering, in AAAI (2014), pp. 1293–1299

  89. C. Tu, W. Zhang, Z. Liu, M. Sun, Max-margin deepwalk: Discriminative learning of network representation, in Proceedings of the 25th International Conference on Artificial Intelligence (AAAI Press, 2016)

  90. V. Venkataraman, P. Srinivasan, Graph embedding aided relationship prediction in heterogeneous networks (CS 512 Project Report, 2016)

  91. K. Verbeek, S. Suri, Metric embedding, hyperbolic space, and social networks, in Proceedings of the thirtieth annual symposium on Computational geometry (ACM, 2014), p. 501

  92. O. Vinyals, A. Toshev, S. Bengio, D. Erhan, Show and tell: A neural image caption generator, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), p. 315

  93. D. Wang, P. Cui, W. Zhu, Structural deep network embedding, in Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2016)

  94. D.R. White, M. Houseman, The navigability of strong ties: Small worlds, tie strength, and network topology, Complexity 8, 72 (2002)

    Article  Google Scholar 

  95. F. Wu, X. Lu, J. Song, S. Yan, Z. M. Zhang, Y. Rui, Y. Zhuang, Learning of multimodal representations with random walks on the click graph, IEEE Trans. Image Proc. 25, 630 (2016)

    Article  ADS  MathSciNet  Google Scholar 

  96. S. Xiang, F. Nie, C. Zhang, C. Zhang, Nonlinear dimensionality reduction with local spline embedding. IEEE Trans, Knowl. Data Eng. 21, 1285 (2009)

    Article  Google Scholar 

  97. L. Xiaoyi, L.H. Du Nan et al., A deep learning approach to link prediction in dynamic networks, in Proceedings of the 2013 SIAM International Conference on Data Mining (Philadelphia, PA, USA: SIAM, 2013)

  98. P. Yanardag, S. Vishwanathan, Deep graph kernels, in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2015), pp. 1365–1374

  99. C. Yang, Z. Liu. Comprehend deepwalk as matrix factorization, arXiv:1501.00358 (2015)

  100. C. Yang, Z. Liu, D. Zhao, M. Sun, E.Y. Chang, Network representation learning with rich text information, in Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina (2015), p. 2111

  101. Z. Yang, W. Cohen, R. Salakhutdinov, Revisiting semi-supervised learning with graph embeddings, in ICML 2016 (2016)

  102. Z. Yang, J. Tang, W. Cohen, Multi-modal bayesian embeddings for learning social knowledge graphs, in Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) (AAAI Press, 2016)

  103. S. Zhai, Z.M. Zhang, Dropout Training of Matrix Factorization and Autoencoder for Link Prediction in Sparse Graphs, (SIAM, 2015), Chap. 51, pp. 451–459

  104. X. Zhao, A. Sala, H. Zheng, B.Y. Zhao, Efficient shortest paths on massive social graphs, in Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom), 2011 7th International Conference on (IEEE, 2011), pp. 77–86

Download references

Author information

Authors and Affiliations

  1. Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) and Facultad de Ciencias Exactas y Naturales, Universidad Nacional de Cuyo, Mendoza, Argentina

    Luis G. Moyano

Authors
  1. Luis G. Moyano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luis G. Moyano.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moyano, L.G. Learning network representations. Eur. Phys. J. Spec. Top. 226, 499–518 (2017). https://doi.org/10.1140/epjst/e2016-60266-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1140/epjst/e2016-60266-2

Search

Navigation