Backdoor Attack in Prompt-Based Continual Learning
arxiv(2024)
Abstract
Prompt-based approaches offer a cutting-edge solution to data privacy issues
in continual learning, particularly in scenarios involving multiple data
suppliers where long-term storage of private user data is prohibited. Despite
delivering state-of-the-art performance, its impressive remembering capability
can become a double-edged sword, raising security concerns as it might
inadvertently retain poisoned knowledge injected during learning from private
user data. Following this insight, in this paper, we expose continual learning
to a potential threat: backdoor attack, which drives the model to follow a
desired adversarial target whenever a specific trigger is present while still
performing normally on clean samples. We highlight three critical challenges in
executing backdoor attacks on incremental learners and propose corresponding
solutions: (1) Transferability: We employ a surrogate dataset and
manipulate prompt selection to transfer backdoor knowledge to data from other
suppliers; (2) Resiliency: We simulate static and dynamic states of the
victim to ensure the backdoor trigger remains robust during intense incremental
learning processes; and (3) Authenticity: We apply binary cross-entropy
loss as an anti-cheating factor to prevent the backdoor trigger from devolving
into adversarial noise. Extensive experiments across various benchmark datasets
and continual learners validate our continual backdoor framework, achieving up
to 100% attack success rate, with further ablation studies confirming our
contributions' effectiveness.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined