Learning Prompt with Distribution-Based Feature Replay for Few-Shot Class-Incremental Learning
arxiv(2024)
摘要
Few-shot Class-Incremental Learning (FSCIL) aims to continuously learn new
classes based on very limited training data without forgetting the old ones
encountered. Existing studies solely relied on pure visual networks, while in
this paper we solved FSCIL by leveraging the Vision-Language model (e.g., CLIP)
and propose a simple yet effective framework, named Learning Prompt with
Distribution-based Feature Replay (LP-DiF). We observe that simply using CLIP
for zero-shot evaluation can substantially outperform the most influential
methods. Then, prompt tuning technique is involved to further improve its
adaptation ability, allowing the model to continually capture specific
knowledge from each session. To prevent the learnable prompt from forgetting
old knowledge in the new session, we propose a pseudo-feature replay approach.
Specifically, we preserve the old knowledge of each class by maintaining a
feature-level Gaussian distribution with a diagonal covariance matrix, which is
estimated by the image features of training images and synthesized features
generated from a VAE. When progressing to a new session, pseudo-features are
sampled from old-class distributions combined with training images of the
current session to optimize the prompt, thus enabling the model to learn new
knowledge while retaining old knowledge. Experiments on three prevalent
benchmarks, i.e., CIFAR100, mini-ImageNet, CUB-200, and two more challenging
benchmarks, i.e., SUN-397 and CUB-200^* proposed in this paper showcase the
superiority of LP-DiF, achieving new state-of-the-art (SOTA) in FSCIL. Code is
publicly available at https://github.com/1170300714/LP-DiF.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要