Advertisement

Data Mining and Knowledge Discovery

, Volume 33, Issue 6, pp 1953–1980 | Cite as

Attributed network embedding via subspace discovery

  • Daokun ZhangEmail author
  • Jie Yin
  • Xingquan Zhu
  • Chengqi Zhang
Article

Abstract

Network embedding aims to learn a latent, low-dimensional vector representations of network nodes, effective in supporting various network analytic tasks. While prior arts on network embedding focus primarily on preserving network topology structure to learn node representations, recently proposed attributed network embedding algorithms attempt to integrate rich node content information with network topological structure for enhancing the quality of network embedding. In reality, networks often have sparse content, incomplete node attributes, as well as the discrepancy between node attribute feature space and network structure space, which severely deteriorates the performance of existing methods. In this paper, we propose a unified framework for attributed network embedding–attri2vec—that learns node embeddings by discovering a latent node attribute subspace via a network structure guided transformation performed on the original attribute space. The resultant latent subspace can respect network structure in a more consistent way towards learning high-quality node representations. We formulate an optimization problem which is solved by an efficient stochastic gradient descent algorithm, with linear time complexity to the number of nodes. We investigate a series of linear and non-linear transformations performed on node attributes and empirically validate their effectiveness on various types of networks. Another advantage of attri2vec is its ability to solve out-of-sample problems, where embeddings of new coming nodes can be inferred from their node attributes through the learned mapping function. Experiments on various types of networks confirm that attri2vec is superior to state-of-the-art baselines for node classification, node clustering, as well as out-of-sample link prediction tasks. The source code of this paper is available at https://github.com/daokunzhang/attri2vec.

Keywords

Network embedding Attributed network Subspace Out-of-sample 

Notes

Acknowledgements

The work is supported by the US National Science Foundation (NSF) through Grant IIS-1763452, and the Australian Research Council (ARC) through Grant LP160100630 and DP180100966. Daokun Zhang is supported by China Scholarship Council (CSC) with No. 201506300082 and a supplementary postgraduate scholarship from CSIRO.

References

  1. Bianconi G, Pin P, Marsili M (2009) Assessing the relevance of node features for network structure. Proc Natl Acad Sci 106(28):11433–11438CrossRefGoogle Scholar
  2. Cao S, Lu W, Xu Q (2015) GraRep: learning graph representations with global structural information. In: Proceedings of the 24th ACM international conference on information and knowledge management. ACM, pp 891–900Google Scholar
  3. Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph representations. In: Proceedings of the 30th AAAI conference on artificial intelligence. AAAI Press, pp 1145–1152Google Scholar
  4. Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) LIBLINEAR: a library for large linear classification. J Mach Learn Res 9(Aug):1871–1874zbMATHGoogle Scholar
  5. Grover A, Leskovec J (2016) node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM sigkdd international conference on knowledge discovery and data mining. ACM, pp 855–864Google Scholar
  6. Guo T, Pan S, Zhu X, Zhang C (2018) CFOND: consensus factorization for co-clustering networked data. IEEE Trans Knowl Data Eng 31:706–719 CrossRefGoogle Scholar
  7. Gutmann MU, Hyvärinen A (2012) Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. J Mach Learn Res 13(Feb):307–361MathSciNetzbMATHGoogle Scholar
  8. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034Google Scholar
  9. Hotelling H (1936) Relations between two sets of variates. Biometrika 28(3/4):321–377CrossRefGoogle Scholar
  10. Huang X, Li J, Hu X (2017a) Accelerated attributed network embedding. In: Proceedings of the 2017 SIAM international conference on data mining. SIAM, pp 633–641Google Scholar
  11. Huang X, Li J, Hu X (2017b) Label informed attributed network embedding. In: Proceedings of the 10th ACM international conference on web search and data mining. ACM, pp 731–739Google Scholar
  12. Kuang D, Ding C, Park H (2012) Symmetric nonnegative matrix factorization for graph clustering. In: Proceedings of the 2012 SIAM international conference on data mining. SIAM, pp 106–117Google Scholar
  13. Leskovec J, Mcauley JJ (2012) Learning to discover social circles in ego networks. In: Advances in neural information processing systems, pp 539–547Google Scholar
  14. Levy O, Goldberg Y (2014) Neural word embedding as implicit matrix factorization. In: Advances in neural information processing systems, pp 2177–2185Google Scholar
  15. Li AQ, Ahmed A, Ravi S, Smola AJ (2014) Reducing the sampling complexity of topic models. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 891–900Google Scholar
  16. Li J, Zhu J, Zhang B (2016) Discriminative deep random walk for network classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics, vol 1, pp 1004–1013Google Scholar
  17. Liao L, He X, Zhang H, Chua TS (2018) Attributed social network embedding. IEEE Trans Knowl Data Eng 30:2257–2270CrossRefGoogle Scholar
  18. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119Google Scholar
  19. Natarajan N, Dhillon IS (2014) Inductive matrix completion for predicting gene-disease associations. Bioinformatics 30(12):i60–i68CrossRefGoogle Scholar
  20. Newman ME (2006) Finding community structure in networks using the eigenvectors of matrices. Phys Rev E 74(3):036104MathSciNetCrossRefGoogle Scholar
  21. Pan S, Wu J, Zhu X, Zhang C, Wang Y (2016) Tri-party deep network representation. In: Proceedings of the 25th international joint conference on artificial intelligence, pp 1895–1901Google Scholar
  22. Perozzi B, Al-Rfou R, Skiena S (2014) DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 701–710Google Scholar
  23. Rahimi A, Recht B (2008) Random features for large-scale kernel machines. In: Advances in neural information processing systems, pp 1177–1184Google Scholar
  24. Reagans R, McEvily B (2003) Network structure and knowledge transfer: the effects of cohesion and range. Adm Sci Q 48(2):240–267 CrossRefGoogle Scholar
  25. Strehl A, Ghosh J (2002) Cluster ensembles—a knowledge reuse framework for combining multiple partitions. J Mach Learn Res 3(Dec):583–617MathSciNetzbMATHGoogle Scholar
  26. Subbaraj K, Sundan B (2015) What happens next? Prediction of disastrous links in covert networks. Disaster Adv 8:53–60Google Scholar
  27. Tang J, Qu M, Wang M, Zhang M, Yan J, Mei Q (2015) LINE: large-scale information network embedding. In: Proceedings of the 24th international conference on world wide web. ACM, pp 1067–1077Google Scholar
  28. Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(Dec):3371–3408MathSciNetzbMATHGoogle Scholar
  29. Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 1225–1234Google Scholar
  30. Wang X, Cui P, Wang J, Pei J, Zhu W, Yang S (2017) Community preserving network embedding. In: Proceedings of the 31st AAAI conference on artificial intelligence, pp 203–209Google Scholar
  31. Yang C, Liu Z, Zhao D, Sun M, Chang EY (2015) Network representation learning with rich text information. In: Proceedings of the 24th international joint conference on artificial intelligence, pp 2111–2117Google Scholar
  32. Yang D, Wang S, Li C, Zhang X, Li Z (2017) From properties to links: deep network embedding on incomplete graphs. In: Proceedings of the 2017 ACM on conference on information and knowledge management. ACM, pp 367–376Google Scholar
  33. Zhang D, Yin J, Zhu X, Zhang C (2016a) Collective classification via discriminative matrix factorization on sparsely labeled networks. In: Proceedings of the 25th ACM international conference on information and knowledge management. ACM, pp 1563–1572Google Scholar
  34. Zhang D, Yin J, Zhu X, Zhang C (2016b) Homophily, structure, and content augmented network representation learning. In: Proceedings of the 16th IEEE international conference on data mining. IEEE, pp 609–618Google Scholar
  35. Zhang D, Yin J, Zhu X, Zhang C (2017) User profile preserving social network embedding. In: Proceedings of the 26th international joint conference on artificial intelligence, pp 3378–3384Google Scholar
  36. Zhang D, Yin J, Zhu X, Zhang C (2018) Network representation learning: a survey. IEEE Trans Big Data (in press)Google Scholar

Copyright information

© The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Centre for Artificial Intelligence, FEITUniversity of Technology SydneyUltimoAustralia
  2. 2.Discipline of Business AnalyticsThe University of SydneySydneyAustralia
  3. 3.Department of CEECSFlorida Atlantic UniversityBoca RatonUSA

Personalised recommendations