Advertisement

A Simple Approach to Attributed Graph Embedding via Enhanced Autoencoder

  • Nasrullah SheikhEmail author
  • Zekarias T. KefatoEmail author
  • Alberto Montresor
Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 881)

Abstract

Network Representation Learning (NRL) aims at learning a low-dimensional representation of nodes in a graph such that its properties are preserved in the learned embedding. NRL methods may exploit different sources of information such as the structural or attribute information of the graph. Recent efforts have shown that jointly using both structure and attributes helps in learning a better representation. Most of these methods rely on highly complex procedures, such as sampling, which makes them non-scalable to large graphs. In this paper, we propose a simple and scalable deep neural network model that learns an embedding by jointly incorporating the network structure and the attribute information. Specifically, the model employs an enhanced decoder that preserves global network structure and also handles the non-linearities of both the network structure and network attributes. We discuss node classification, link prediction, and network reconstruction experiments on four real-world datasets, demonstrating that our approach achieves better performance against the state-of-the-art baselines.

Keywords

Attributed graphs Network embedding Unsupervised Learning 

References

  1. 1.
    Backstrom, L., Leskovec, J.: Supervised random walks: predicting and recommending links in social networks. In: Proceedings of the 4th ACM International Conference on Web Search and Data Mining, pp. 635–644. ACM (2011)Google Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Proceedings of the 14th International Conference on Neural Information Processing Systems, NIPS 2001, pp. 585–591. MIT Press (2001)Google Scholar
  3. 3.
    Fouss, F., Pirotte, A., Renders, J.-M., Saerens, M.: Random-walk computation of similarities between nodes of a graph with application to collaborative recommendation. IEEE Trans. Knowl. Data Eng. 19(3), 355–369 (2007)CrossRefGoogle Scholar
  4. 4.
    Gao, H., Huang, H.: Deep attributed network embedding. In: Proceedings of the 27th International Conference on Artificial Intelligence, IJCAI 2018, pp. 3364–3370 (2018)Google Scholar
  5. 5.
    Grover, A., Leskovec, J.: Node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pp. 855–864. ACM (2016)Google Scholar
  6. 6.
    Hamilton, W., Ying, R., Leskovec, J.: Inductive representation learning on large graphs. CoRR, abs/1706.02216 (2017)Google Scholar
  7. 7.
    Huang, X., Li, J., Hu, X.: Accelerated attributed network embedding. In: SIAM International Conference on Data Mining, pp. 633–641 (2017)CrossRefGoogle Scholar
  8. 8.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR, abs/1412.6980 (2014)Google Scholar
  9. 9.
    Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. CoRR, abs/1609.02907 (2016)Google Scholar
  10. 10.
    Kipf, T.N., Welling, M.: Variational graph auto-encoders. In: NIPS Workshop on Bayesian Deep Learning (2016)Google Scholar
  11. 11.
    Le, Q.V., Mikolov, T.: Distributed representations of sentences and documents. CoRR (2014)Google Scholar
  12. 12.
    Liben-Nowell, D., Kleinberg, J.: The link-prediction problem for social networks. J. Am. Soc. Inf. Sci. Technol. 58(7), 1019–1031 (2007)CrossRefGoogle Scholar
  13. 13.
    Luo, D., Ding, C.H.Q., Nie, F., Huang, H.: Cauchy graph embedding. In: Proceedings of the 28th International Conference on Machine Learning, ICML 2011, pp. 553–560 (2011)Google Scholar
  14. 14.
    Lü, L., Zhou, T.: Link prediction in complex networks: a survey. Phys. A 390(6), 11501170 (2011)CrossRefGoogle Scholar
  15. 15.
    Meng, Z., Liang, S., Bao, H., Zhang, X.: Co-embedding attributed networks. In: Proceedings of the 12th ACM International Conference on Web Search and Data Mining, WSDM 2019. ACM (2019)Google Scholar
  16. 16.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR, abs/1301.3781 (2013)Google Scholar
  17. 17.
    Pan, S., Wu, J., Zhu, X., Zhang, C., Wang, Y.: Tri-party deep network representation. In: Proceedings of the 25th International Conference on Artificial Intelligence, IJCAI 2016, pp. 1895–1901. AAAI Press (2016)Google Scholar
  18. 18.
    Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2014, pp. 701–710. ACM (2014)Google Scholar
  19. 19.
    Sen, P., Namata, G., Bilgic, M., Getoor, L., Gallagher, B., Eliassi-Rad, T.: Collective classification in network data. AI Mag. 29(3), 93–106 (2008)CrossRefGoogle Scholar
  20. 20.
    Sheikh, N., Kefato, Z., Montresor, A.: GAT2VEC: representation learning for attributed graphs. Computing 101(3), 187–209 (2018)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Tang, J., Qu, M., Mei, Q.: PTE: predictive text embedding through large-scale heterogeneous text networks. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2015, pp. 1165–1174. ACM (2015)Google Scholar
  22. 22.
    Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: LINE: large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, WWW 2015, pp. 1067–1077 (2015)Google Scholar
  23. 23.
    Tang, L., Liu, H.: Relational learning via latent social dimensions. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2009, pp. 817–826. ACM (2009)Google Scholar
  24. 24.
    Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319 (2000)CrossRefGoogle Scholar
  25. 25.
    Wang, D., Cui, P., Zhu, W.: Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pp. 1225–1234. ACM (2016)Google Scholar
  26. 26.
    Yang, C., Liu, Z., Zhao, D., Sun, M., Chang, E.: Network representation learning with rich text information. In: Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI 2015, pp. 2111–2117. AAAI Press (2015)Google Scholar
  27. 27.
    Yang, Z., Cohen, W., Salakhutdinov, W.: Revisiting semi-supervised learning with graph embeddings. CoRR, abs/1603.08861 (2016)Google Scholar
  28. 28.
    Zhang, Z., Yang, H., Bu, J., Zhou, S., Yu, P., Zhang, J., Ester, M., Wang, C.: ANRL: attributed network representation learning via deep neural networks. In: Proceedings of the 27th International Conference on Artificial Intelligence, IJCAI 2018, pp. 3155–3161 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of TrentoTrentoItaly
  2. 2.Royal Institute of TechnologyStockholmSweden

Personalised recommendations