Advertisement

Network Representation Learning Using Local Sharing and Distributed Matrix Factorization (LSDMF)

  • Pradumn Kumar Pandey
Conference paper
Part of the Springer Proceedings in Complexity book series (SPCOM)

Abstract

Vector embedding over a real network is considered as feature learning of nodes of the network which is utilized in many downstream machine learning applications such as link prediction. A network of size n can be represented as a collection of n vectors (feature vectors) of dimension d (≪ n) which have encoded structural and spectral information of the associated network. These feature vectors can be used in two ways: first, in the extraction of existing links and other higher order structural or functional relations among the nodes of the network and second, in the prediction of the structural evolution of the network in near future. It is observed that matrix factorization based vector embedding algorithms are able to learn more informative feature vectors but scalability is a major bottleneck due to memory and computationally intensive task.

In this paper, we present a novel distributed algorithm to learn feature vectors. It is considered that a node stores one feature vector of one of its neighbours known as shared vector, along with its own feature vector. And the learning of feature vector of a node includes only feature vectors and shared vectors stored in its neighbourhood only. Feature vectors get updated during the learning, so is shared vectors. Hence, a local sharing phenomenon leads to sharing of global information dynamically. The proposed distributed algorithm learns matrix factorization of a given network in which a node only utilizes the information available at its neighbouring nodes and connected nodes exchange feature vectors dynamically. Thus, the proposed algorithm doesn’t have the limitation on its scalability. The performance of the proposed distributed algorithm for network representation learning (NRL) is evaluated for the learning of first-order proximity, spectral distance, and link prediction. The proposed network representation learning algorithm outperforms the existing state-of-the-art NRL algorithms such as node2vec, deep walk, and edge-based matrix factorization.

Keywords

Distributed matrix factorization Local sharing Graph embedding First-order proximity Link prediction Network representation learning 

References

  1. 1.
    Ahmed, A., Shervashidze, N., Narayanamurthy, S., Josifovski, V., Smola, A.J.: Distributed large-scale natural graph factorization. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 37–48. ACM, New York (2013)Google Scholar
  2. 2.
    Barabási, A.L., Albert, R.: Emergence of scaling in random networks. Science 286(5439), 509–512 (1999)ADSMathSciNetCrossRefGoogle Scholar
  3. 3.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in Neural Information Processing Systems, pp. 585–591 (2002)Google Scholar
  4. 4.
    Boguná, M., Pastor-Satorras, R., Díaz-Guilera, A., Arenas, A.: Models of social networks based on social distance attachment. Phys. Rev. E 70(5), 056122 (2004)ADSCrossRefGoogle Scholar
  5. 5.
    Bornholdt, S., Schuster, H.G.: Handbook of Graphs and Networks: From the Genome to the Internet. Wiley, Weinheim (2006)zbMATHGoogle Scholar
  6. 6.
    Bu, D., Zhao, Y., Cai, L., Xue, H., Zhu, X., Lu, H., Zhang, J., Sun, S., Ling, L., Zhang, N., et al.: Topological structure analysis of the protein–protein interaction network in budding yeast. Nucleic Acids Res. 31(9), 2443–2450 (2003)CrossRefGoogle Scholar
  7. 7.
    Butler, S., Chung, F.: Spectral graph theory. In: Handbook of Linear Algebra, p. 47. CRC Press, Boca Raton (2006)Google Scholar
  8. 8.
    Cao, S., Lu, W., Xu, Q.: Grarep: learning graph representations with global structural information. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 891–900. ACM, New York (2015)Google Scholar
  9. 9.
    Cazals, F., Karande, C.: A note on the problem of reporting maximal cliques. Theor. Comput. Sci. 407(1–3), 564–568 (2008)MathSciNetCrossRefGoogle Scholar
  10. 10.
    De, A., Bhattacharya, S., Sarkar, S., Ganguly, N., Chakrabarti, S.: Discriminative link prediction using local, community, and global signals. IEEE Trans. Knowl. Data Eng. 28(8), 2057–2070 (2016)CrossRefGoogle Scholar
  11. 11.
    Ferber, J.: Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence, vol. 1. Addison-Wesley, Reading (1999)Google Scholar
  12. 12.
    Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864. ACM, New York (2016)Google Scholar
  13. 13.
    Guimera, R., Danon, L., Diaz-Guilera, A., Giralt, F., Arenas, A.: Self-similar community structure in a network of human interactions. Phys. Rev. E 68(6), 065103 (2003)ADSCrossRefGoogle Scholar
  14. 14.
    Hamilton, W.L., Ying, R., Leskovec, J.: Representation learning on graphs: methods and applications. (2017, preprint). arXiv:1709.05584Google Scholar
  15. 15.
    Jackson, M.O.: Social and Economic Networks. Princeton University Press, Princeton (2010)CrossRefGoogle Scholar
  16. 16.
    Leskovec, J., Krevl, A.: SNAP datasets: Stanford large network dataset collection (2014). http://snap.stanford.edu/data
  17. 17.
    Leskovec, J., Kleinberg, J., Faloutsos, C.: Graph evolution: densification and shrinking diameters. ACM Trans. Knowl. Discov. Data 1(1), 2 (2007)CrossRefGoogle Scholar
  18. 18.
    Levy, O., Goldberg, Y.: Dependency-based word embeddings. In: Association for Computational Linguistics ACL, vol. 2, pp. 302–308 (2014)Google Scholar
  19. 19.
    Newman, M.E.: Complex systems: a survey. (2011, preprint). arXiv:1112.1440Google Scholar
  20. 20.
    Newman, M.E., Girvan, M.: Finding and evaluating community structure in networks. Phys. Rev. E 69(2), 026113 (2004)ADSCrossRefGoogle Scholar
  21. 21.
    Ou, M., Cui, P., Pei, J., Zhang, Z., Zhu, W.: Asymmetric transitivity preserving graph embedding. In: Knowledge Discovery and Data Mining (KDD), pp. 1105–1114 (2016)Google Scholar
  22. 22.
    Pandey, P.K., Adhikari, B.: Context dependent preferential attachment model for complex networks. Phys. A Stat. Mech. Appl. 436, 499–508 (2015)CrossRefGoogle Scholar
  23. 23.
    Pandey, P.K., Adhikari, B.: A parametric model approach for structural reconstruction of scale-free networks. IEEE Trans. Knowl. Data Eng. 29(10), 2072–2085 (2017)CrossRefGoogle Scholar
  24. 24.
    Pandey, P.K., Badarla, V.: Reconstruction of network topology using status-time-series data. Phys. A Stat. Mech. Appl. 490, 573–583 (2018)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710. ACM, New York (2014)Google Scholar
  26. 26.
    Scott, J.: Social Network Analysis. Sage, Thousand Oaks (2017)Google Scholar
  27. 27.
    Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small-world’ networks. Nature 393(6684), 440 (1998)ADSCrossRefGoogle Scholar
  28. 28.
    White, S., Smyth, P.: A spectral clustering approach to finding communities in graphs. In: Proceedings of the 2005 SIAM International Conference on Data Mining, pp. 274–285. SIAM, Philadelphia (2005)Google Scholar
  29. 29.
    Zhang, D., Yin, J., Zhu, X., Zhang, C.: Network representation learning: a survey. IEEE Xplore Digital Library (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Pradumn Kumar Pandey
    • 1
  1. 1.Indian Institute of Technology RoorkeeRoorkeeIndia

Personalised recommendations