Skip to main content
Article
Prototypical Graph Contrastive Learning
IEEE Transactions on Neural Networks and Learning Systems
  • Shuai Lin, School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
  • Liu Chen, School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
  • Pan Zhou, Sea AI Laboratory, Singapore, Singapore
  • Zi-Yuan Hu, School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
  • Shuojia Wang, Tencent Jarvis Lab, Shenzhen, China
  • Ruihui Zhao, Tencent Jarvis Lab, Shenzhen, China
  • Yefeng Zheng, Tencent Jarvis Lab, Shenzhen, China
  • Liang Lin, School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
  • Eric Xing, Mohamed bin Zayed University of Artificial Intelligence
  • Xiaodan Liang, School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China
Document Type
Article
Abstract

Graph-level representations are critical in various real-world applications, such as predicting the properties of molecules. However, in practice, precise graph annotations are generally very expensive and time-consuming. To address this issue, graph contrastive learning constructs an instance discrimination task, which pulls together positive pairs (augmentation pairs of the same graph) and pushes away negative pairs (augmentation pairs of different graphs) for unsupervised representation learning. However, since for a query, its negatives are uniformly sampled from all graphs, existing methods suffer from the critical sampling bias issue, i.e., the negatives likely having the same semantic structure with the query, leading to performance degradation. To mitigate this sampling bias issue, in this article, we propose a prototypical graph contrastive learning (PGCL) approach. Specifically, PGCL models the underlying semantic structure of the graph data via clustering semantically similar graphs into the same group and simultaneously encourages the clustering consistency for different augmentations of the same graph. Then, given a query, it performs negative sampling via drawing the graphs from those clusters that differ from the cluster of query, which ensures the semantic difference between query and its negative samples. Moreover, for a query, PGCL further reweights its negative samples based on the distance between their prototypes (cluster centroids) and the query prototype such that those negatives having moderate prototype distance enjoy relatively large weights. This reweighting strategy is proven to be more effective than uniform sampling. Experimental results on various graph benchmarks testify the advantages of our PGCL over state-of-the-art methods. The code is publicly available at https://github.com/ha-lins/PGCL. IEEE

DOI
10.1109/TNNLS.2022.3191086
Publication Date
7-27-2022
Keywords
  • Contrastive learning,
  • graph representation learning,
  • Kernel,
  • Loss measurement,
  • Perturbation methods,
  • Prototypes,
  • Representation learning,
  • self-supervised learning,
  • Semantics,
  • Task analysis,
  • Data mining,
  • Graphic methods,
  • Job analysis,
  • Learning systems,
  • Perturbation techniques
Comments

IR Deposit conditions:

OA version (pathway a) Accepted version

No embargo

When accepted for publication, set statement to accompany deposit (see policy)

Must link to publisher version with DOI

Publisher copyright and source must be acknowledged

Citation Information
S. Lin et al., "Prototypical Graph Contrastive Learning," in IEEE Transactions on Neural Networks and Learning Systems, 2022, doi: 10.1109/TNNLS.2022.3191086.