Towards Robustness and Diversity: Continual Learning in Dialog Generation with Text-Mixup and Batch Nuclear-Norm Maximization
arxiv(2024)
摘要
In our dynamic world where data arrives in a continuous stream, continual
learning enables us to incrementally add new tasks/domains without the need to
retrain from scratch. A major challenge in continual learning of language model
is catastrophic forgetting, the tendency of models to forget knowledge from
previously trained tasks/domains when training on new ones. This paper studies
dialog generation under the continual learning setting. We propose a novel
method that 1) uses Text-Mixup as data augmentation to avoid model
overfitting on replay memory and 2) leverages Batch-Nuclear Norm Maximization
(BNNM) to alleviate the problem of mode collapse. Experiments on a 37-domain
task-oriented dialog dataset and DailyDialog (a 10-domain chitchat dataset)
demonstrate that our proposed approach outperforms the state-of-the-art in
continual learning.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要