Neighborhood-enhanced contrast for pre-training graph neural networks

Abstract

Pre-training graph neural networks (GNNs) have been proposed to promote graph-related downstream tasks, such as link prediction and node classification. Most existing works employ contrastive learning to explore graph characteristics by enforcing positive sample pairs to be close and negative sample pairs to be distant after performing data augmentation on the input graph. However, these methods apply random operations on input data in data augmentation and sample pairs construction, which leads to neglecting central nodes and the neighbor relationship between nodes. To address the corresponding problem, we propose a novel framework for pre-training GNNs, named Neighborhood-Enhanced Contrast for Pre-Training Graph Neural Networks (NECPT). Specifically, we propose data augmentation strategy based on node centrality to preserve central nodes and corresponding edges, which is used to generate two semantically similar views from input graph. Notably, our NECPT constructs sample pairs by integrating the potential node neighbors in graph structure and semantic space to explore general graph regularities. After generating node representations with GNN encoders and multilayer perceptrons, contrastive sample pairs are selected from different node neighbors, which combines diverse neighborhood relations into contrastive learning. Finally, node representations obtained from the model are used to predict the attributes of nodes and edges, which extracts deep semantic connections between attribute and structure information. Extensive experiments on benchmark datasets in biology and chemistry demonstrate the effectiveness of our proposed approach.