Advertisement

A Graph Representation Learning Algorithm Based on Attention Mechanism and Node Similarity

  • Kun Guo
  • Deqin Wang
  • Jiangsheng Huang
  • Yuzhong ChenEmail author
  • Zhihao Zhu
  • Jianning Zheng
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1042)

Abstract

Recently graph representation learning has attracted much attention of researchers, aiming to capture and preserve the graph structure by encoding it into low-dimensional vectors. Attention mechanism is a recent research hotspot in learning the representation of graph. In this paper, a graph representation learning algorithm based on Attention Mechanism and Node Similarity (AMNS for short) is proposed. Firstly, the similarity neighborhood is generated for each node in graph. Secondly, attention mechanism is used to learn weight coefficients for each node and its similarity neighborhood. Thirdly, the node vectors are generated by aggregating its similarity neighborhood with weight coefficients. Finally, node vectors are applied to many tasks, e.g., node classification and clustering. The experiments on real-world network datasets prove that the AMNS algorithm achieves excellent results.

Keywords

Graph representation learning Attention mechanism Node similarity 

Notes

Acknowledgements

This work is partly supported by the National Natural Science Foundation of China under Grant No. 61300104, No. 61300103 and No. 61672159, the Fujian Province High School Science Fund for Distinguished Young Scholars under Grant No. JA12016, the Fujian Natural Science Funds for Distinguished Young Scholar under Grant No. 2015J06014, the Fujian Industry-Academy Cooperation Project under Grant No. 2017H6008 and No. 2018H6010, and Haixi Government Big Data Application Cooperative Innovation Center.

References

  1. 1.
    Bhagat, S., Cormode, G., Muthukrishnan, S.: Node classification in social networks. In: Aggarwal, C. (ed.) Social Network Data Analytics, pp. 115–148. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-1-4419-8462-3_5CrossRefGoogle Scholar
  2. 2.
    Malliaros, F.D., Vazirgiannis, M.: Clustering and community detection in directed networks: a survey. Phys. Rep. 533(4), 95–142 (2013).  https://doi.org/10.1016/j.physrep.2013.08.002MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Resnick, P., Varian, H.: Recommender systems. Commun. ACM 40(3), 56–59 (1997).  https://doi.org/10.1145/245108.245121CrossRefGoogle Scholar
  4. 4.
    Parthasarathy, S., Ruan, Y., Satuluri, V.: Community discovery in social networks: applications, methods and emerging trends. In: Aggarwal, C. (ed.) Social network data analytics, pp. 79–113. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-1-4419-8462-3_4CrossRefGoogle Scholar
  5. 5.
    Hamilton, W.L., Ying, R., Leskovec, J.: Representation learning on graphs: methods and applications. arXiv preprint arXiv:1709.05584 (2017)
  6. 6.
    Perozzi, B., Al-Rfou, R., Skiena, S: DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710. ACM (2014).  https://doi.org/10.1145/2623330.2623732
  7. 7.
    Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864. ACM (2016).  https://doi.org/10.1145/2939672.2939754
  8. 8.
    Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: large-scale information network embedding. In: Proceedings of the 24th International Conference on World Wide Web, pp. 1067–1077. International World Wide Web Conferences Steering Committee (2015).  https://doi.org/10.1145/2736277.2741093
  9. 9.
    Wang, D., Cui, P., Zhu, W.: Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1225–1234. ACM (2016).  https://doi.org/10.1145/2939672.2939753
  10. 10.
    Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2009).  https://doi.org/10.1109/TNN.2008.2005605CrossRefGoogle Scholar
  11. 11.
    Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
  12. 12.
    Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)Google Scholar
  13. 13.
    Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
  14. 14.
    Lu, W., Janssen, J., Milios, E., Japkowicz, N., Zhang, Y.: Node similarity in the citation graph. Knowl. Inf. Syst. 11(1), 105–129 (2007).  https://doi.org/10.1007/s10115-006-0023-9CrossRefGoogle Scholar
  15. 15.
    Thekumparampil, K.K., et al.: Attention-based graph neural network for semi-supervised learning. arXiv preprint arXiv:1803.03735 (2018)
  16. 16.
    Abu-El-Haija, S., et al.: Watch your step: learning node embeddings via graph attention. In: Advances in Neural Information Processing Systems, pp. 9180–9190 (2018)Google Scholar
  17. 17.
    Niwattanakul, S., Singthongchai, J., Naenudorn, E., Wanapu, S.: Using of Jaccard coefficient for keywords similarity. In: Proceedings of the International Multiconference of Engineers and Computer Scientists, pp. 380–384 (2013)Google Scholar
  18. 18.
    Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., Eliassi-Rad, T.: Collective classification in network data. AI Mag. 29(3), 93 (2008)CrossRefGoogle Scholar
  19. 19.
    Tang, L., Liu, H.: Relational learning via latent social dimensions. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 817–826. ACM (2009).  https://doi.org/10.1145/1557019.1557109
  20. 20.
    Danon, L., Diaz-Guilera, A., Duch, J., Arenas, A.: Comparing community structure identification. J. Stat. Mech: Theory Exp. 2005(09), P09008 (2005)CrossRefGoogle Scholar
  21. 21.
    Maaten, L.V.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)zbMATHGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Kun Guo
    • 1
    • 2
    • 3
  • Deqin Wang
    • 1
    • 2
    • 3
  • Jiangsheng Huang
    • 4
  • Yuzhong Chen
    • 1
    • 2
    • 3
    Email author
  • Zhihao Zhu
    • 1
  • Jianning Zheng
    • 4
  1. 1.College of Mathematics and Computer SciencesFuzhou UniversityFuzhouChina
  2. 2.Fujian Provincial Key Laboratory of Network Computing and Intelligence Information ProcessingFuzhouChina
  3. 3.Key Laboratory of Spatial Data Mining and Information SharingMinistry of EducationFuzhouChina
  4. 4.Power Science and Technology Corporation State Grid Information and Telecommunication GroupXiamenChina

Personalised recommendations