Balancing stability and plasticity in continual learning: the readout-decomposition of activation change (RDAC) framework
CoRR(2023)
摘要
Continual learning (CL) algorithms strive to acquire new knowledge while
preserving prior information. However, this stability-plasticity trade-off
remains a central challenge. This paper introduces a framework that dissects
this trade-off, offering valuable insights into CL algorithms. The
Readout-Decomposition of Activation Change (RDAC) framework first addresses the
stability-plasticity dilemma and its relation to catastrophic forgetting. It
relates learning-induced activation changes in the range of prior readouts to
the degree of stability and changes in the null space to the degree of
plasticity. In deep non-linear networks tackling split-CIFAR-110 tasks, the
framework clarifies the stability-plasticity trade-offs of the popular
regularization algorithms Synaptic intelligence (SI), Elastic-weight
consolidation (EWC), and learning without Forgetting (LwF), and replay-based
algorithms Gradient episodic memory (GEM), and data replay. GEM and data replay
preserved stability and plasticity, while SI, EWC, and LwF traded off
plasticity for stability. The inability of the regularization algorithms to
maintain plasticity was linked to them restricting the change of activations in
the null space of the prior readout. Additionally, for one-hidden-layer linear
neural networks, we derived a gradient decomposition algorithm to restrict
activation change only in the range of the prior readouts, to maintain high
stability while not further sacrificing plasticity. Results demonstrate that
the algorithm maintained stability without significant plasticity loss. The
RDAC framework informs the behavior of existing CL algorithms and paves the way
for novel CL approaches. Finally, it sheds light on the connection between
learning-induced activation/representation changes and the stability-plasticity
dilemma, also offering insights into representational drift in biological
systems.
更多查看译文
关键词
continual learning,activation change,readout-decomposition
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要