InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions
CoRR(2024)
Abstract
Instruction tuning effectively optimizes Large Language Models (LLMs) for
downstream tasks. Due to the changing environment in real-life applications,
LLMs necessitate continual task-specific adaptation without catastrophic
forgetting. Considering the heavy computational cost, replay-based Continual
Learning (CL) methods are the simplest and most widely used for LLMs to address
the forgetting issue. However, traditional replay-based methods do not fully
utilize instructions to customize the replay strategy. In this work, we propose
a novel paradigm called Instruction-based Continual Learning (InsCL). InsCL
dynamically replays previous data based on task similarity, calculated by
Wasserstein Distance with instructions. Moreover, we further introduce an
Instruction Information Metric (InsInfo) to quantify the complexity and
diversity of instructions. According to InsInfo, InsCL guides the replay
process more inclined to high-quality data. We conduct extensive experiments
over 16 tasks with different training orders, observing consistent performance
improvements of InsCL. When all tasks have been trained, InsCL achieves
performance gains of 3.0 Relative Gain compared with Random Replay, and 27.96
Relative Gain compared with No Replay.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined