Abstract
Graph contrastive learning has attracted considerable attention and made remarkable progress in node representation learning and clustering for attributed graphs. However, existing contrastive-based clustering methods separate the processes of node representation learning and graph clustering into two stages, making it difficult to ensure good clustering. Therefore, it remains a challenge to design an effective contrastive learning method that jointly optimizes node representations and graph clustering. Moreover, existing random augmentation strategies to generate contrastive views may destroy the original topological structures of clusters in graphs. So it is crucial to construct an augmented graph that preserves the cluster structure of a given graph while benefitting graph clustering. To address these problems, we propose a contrastive learning method with cluster-preserving augmentation for attributed graph clustering. Specifically, we construct a contrasting view based on the generated kNN graph and edge betweenness centrality to preserve the inherent cluster structure of a graph. Then, a multilevel contrastive mechanism is proposed to maximize the agreement between node representations in multiple latent spaces. Finally, the objective of node representation learning is jointly optimized with the self-supervised clustering objective to obtain cluster distributions and discriminative node representations simultaneously. Extensive experiments on seven widely used real-world graphs demonstrate that the proposed model consistently outperforms existing state-of-the-art methods on clustering tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The code is available at https://github.com/Zhengymm/CCA-AGC.
References
Bo, D., Wang, X., Shi, C., Zhu, M., Lu, E., Cui, P.: Structural deep clustering network. In: Proceedings of the The Web Conference 2020, pp. 1400–1410 (2020)
Girvan, M., Newman, M.E.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. 99(12), 7821–7826 (2002)
Hafidi, H., Ghogho, M., Ciblat, P., Swami, A.: Negative sampling strategies for contrastive self-supervised learning of graph representations. Signal Process. 190, 108310 (2022)
Hamilton, W.L., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems 30, pp. 1024–1034 (2017)
Hassani, K., Ahmadi, A.H.K.: Contrastive multi-view representation learning on graphs. In: Proceedings of the 37th International Conference on Machine Learning, vol. 119, pp. 4116–4126 (2020)
Hjelm, R.D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., Bengio, Y.: Learning deep representations by mutual information estimation and maximization. In: Proceedings of the 7th International Conference on Learning Representations (2019)
Hou, Z., et al.: Graphmae: Self-supervised masked graph autoencoders. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 594–604 (2022)
Jin, M., Zheng, Y., Li, Y., Gong, C., Zhou, C., Pan, S.: Multi-scale contrastive siamese networks for self-supervised graph representation learning. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence, pp. 1477–1483 (2021)
Jing, B., Park, C., Tong, H.: HDMI: high-order deep multiplex infomax. In: Proceedings of the The Web Conference 2021, pp. 2414–2424 (2021)
Keyvanpour, M.R., Shirzad, M.B., Ghaderi, M.: AD-C: a new node anomaly detection based on community detection in social networks. Int. J. Electron. Bus. 15(3), 199–222 (2020)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: Proceedings of the 2nd International Conference on Learning Representations (2014)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations (2017)
Klicpera, J., Weißenberger, S., Günnemann, S.: Diffusion improves graph learning. In: Advances in Neural Information Processing Systems 32, pp. 13333–13345 (2019)
Lee, N., Lee, J., Park, C.: Augmentation-free self-supervised learning on graphs. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence, pp. 7372–7380 (2022)
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., Tang, J.: Self-supervised learning: Generative or contrastive. IEEE Trans. Knowl. Data Eng. 35(1), 857–876 (2023)
Liu, Y., et al.: Hard sample aware network for contrastive deep graph clustering. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence (2023)
MacQueen, J.B.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297 (1967)
Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Park, J., Lee, M., Chang, H.J., Lee, K., Choi, J.Y.: Symmetric graph convolutional autoencoder for unsupervised graph representation learning. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, pp. 6518–6527 (2019)
Peng, H., et al.: Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In: Proceedings of the 2018 World Wide Web Conference, pp. 1063–1072 (2018)
Peng, Z., et al.: Graph representation learning via graphical mutual information maximization. In: Proceedings of the The Web Conference, pp. 259–270 (2020)
Peng, Z., Liu, H., Jia, Y., Hou, J.: Attention-driven graph clustering network. In: Proceedings of the ACM Multimedia Conference, pp. 935–943 (2021)
Peng, Z., Liu, H., Jia, Y., Hou, J.: Deep attention-guided graph clustering with dual self-supervision. IEEE Trans. Circ. Syst. Video Technol. (2022)
Satuluri, V., et al.: Simclusters: community-based representations for heterogeneous recommendations at twitter. In: Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3183–3193 (2020)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)
Shi, W., Rajkumar, R.: Point-gnn: graph neural network for 3d object detection in a point cloud. In: Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1708–1716 (2020)
Sun, F., Hoffmann, J., Verma, V., Tang, J.: Infograph: unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In: Proceedings of the 8th International Conference on Learning Representations (2020)
Trivedi, P., Lubana, E.S., Yan, Y., Yang, Y., Koutra, D.: Augmentations in graph contrastive learning: Current methodological flaws & towards better practices. In: Proceedings of the ACM Web Conference 2022, pp. 1538–1549 (2022)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: Proceedings of 6th International Conference on Learning Representations (2018)
Velickovic, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. In: Proceedings of the 7th International Conference on Learning Representations (2019)
Wang, C., Pan, S., Hu, R., Long, G., Jiang, J., Zhang, C.: Attributed graph clustering: a deep attentional embedding approach. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 3670–3676 (2019)
Wang, C., Pan, S., Yu, C.P., Hu, R., Long, G., Zhang, C.: Deep neighbor-aware embedding for node clustering in attributed graphs. Pattern Recogn. 122, 108230 (2022)
Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications. Cambridge University Press (1994)
Wu, L., Lin, H., Tan, C., Gao, Z., Li, S.Z.: Self-supervised learning on graphs: contrastive, generative, or predictive. IEEE Trans. Knowl. Data Eng. 35(4), 4216–4235 (2023)
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2021)
Xie, J., Girshick, R.B., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: Proceedings of the 33nd International Conference on Machine Learning, vol. 48, pp. 478–487 (2016)
Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations (2019)
Yang, X., et al.: Cluster-guided contrastive graph clustering network. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence (2023)
You, Y., Chen, T., Shen, Y., Wang, Z.: Graph contrastive learning automated. In: Proceedings of the 38th International Conference on Machine Learning, vol. 139, pp. 12121–12132 (2021)
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. In: Advances in Neural Information Processing Systems 33 (2020)
You, Y., Chen, T., Wang, Z., Shen, Y.: Bringing your own view: Graph contrastive learning without prefabricated data augmentations. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining, pp. 1300–1309 (2022)
Zhang, H., Wu, Q., Yan, J., Wipf, D., Yu, P.S.: From canonical correlation analysis to self-supervised graph neural networks. In: Advances in Neural Information Processing Systems 34, pp. 76–89 (2021)
Zhang, Y., Zhu, H., Song, Z., Koniusz, P., King, I.: COSTA: covariance-preserving feature augmentation for graph contrastive learning. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2524–2534 (2022)
Zhao, T., Liu, G., Günnemann, S., Jiang, M.: Graph data augmentation for graph machine learning: a survey. arXiv preprint arXiv:2202.08871 (2022)
Zhao, T., Liu, Y., Neves, L., Woodford, O.J., Jiang, M., Shah, N.: Data augmentation for graph neural networks. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 11015–11023 (2021)
Zhou, J., Xie, C., Wen, Z., Zhao, X., Xuan, Q.: Data augmentation on graphs: a survey. arXiv preprint arXiv:2212.09970 (2022)
Zhu, Y., Xu, Y., Liu, Q., Wu, S.: An empirical study of graph contrastive learning. In: Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1 (2021)
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020)
Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Graph contrastive learning with adaptive augmentation. In: Proceedings of the The Web Conference 2021, pp. 2069–2080 (2021)
Acknowledgments
This work was supported by the National Natural Science Foundation of China (61876016) and the National Key R &D Program of China (2018AAA0100302).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Ethical Statement
Our research aims to introduce a new algorithm for attributed graph clustering. This study does not involve human or animal participants. The data used in this research are obtained from publicly available datasets, and no data (including images) have been fabricated or manipulated to support the conclusions. The authors declare that they have no conflict of interest in this work. All co-authors have agreed to the submission of this version. We certify that this manuscript is original and has not been previously published or submitted elsewhere. The article is also not divided into several parts to increase the number of submissions. We are committed to ensuring the transparency and fairness of our research results and our research are conducted in accordance with relevant ethical standards and principles of academic integrity.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, Y., Jia, C., Yu, J. (2023). Contrastive Learning with Cluster-Preserving Augmentation for Attributed Graph Clustering. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14169. Springer, Cham. https://doi.org/10.1007/978-3-031-43412-9_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-43412-9_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43411-2
Online ISBN: 978-3-031-43412-9
eBook Packages: Computer ScienceComputer Science (R0)
