Contrastive Learning with Negative Sampling Correction
CoRR(2024)
摘要
As one of the most effective self-supervised representation learning methods,
contrastive learning (CL) relies on multiple negative pairs to contrast against
each positive pair. In the standard practice of contrastive learning, data
augmentation methods are utilized to generate both positive and negative pairs.
While existing works have been focusing on improving the positive sampling, the
negative sampling process is often overlooked. In fact, the generated negative
samples are often polluted by positive samples, which leads to a biased loss
and performance degradation. To correct the negative sampling bias, we propose
a novel contrastive learning method named Positive-Unlabeled Contrastive
Learning (PUCL). PUCL treats the generated negative samples as unlabeled
samples and uses information from positive samples to correct bias in
contrastive loss. We prove that the corrected loss used in PUCL only incurs a
negligible bias compared to the unbiased contrastive loss. PUCL can be applied
to general contrastive learning problems and outperforms state-of-the-art
methods on various image and graph classification tasks. The code of PUCL is in
the supplementary file.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要