Learning from True-False Labels via Multi-modal Prompt Retrieving
CoRR(2024)
Abstract
Weakly supervised learning has recently achieved considerable success in
reducing annotation costs and label noise. Unfortunately, existing weakly
supervised learning methods are short of ability in generating reliable labels
via pre-trained vision-language models (VLMs). In this paper, we propose a
novel weakly supervised labeling setting, namely True-False Labels (TFLs) which
can achieve high accuracy when generated by VLMs. The TFL indicates whether an
instance belongs to the label, which is randomly and uniformly sampled from the
candidate label set. Specifically, we theoretically derive a risk-consistent
estimator to explore and utilize the conditional probability distribution
information of TFLs. Besides, we propose a convolutional-based Multi-modal
Prompt Retrieving (MRP) method to bridge the gap between the knowledge of VLMs
and target learning tasks. Experimental results demonstrate the effectiveness
of the proposed TFL setting and MRP learning method. The code to reproduce the
experiments is at https://github.com/Tranquilxu/TMP.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined