Can Similarity-Based Domain-Ordering Reduce Catastrophic Forgetting for Intent Recognition?
CoRR(2024)
摘要
Task-oriented dialogue systems are expected to handle a constantly expanding
set of intents and domains even after they have been deployed to support more
and more functionalities. To live up to this expectation, it becomes critical
to mitigate the catastrophic forgetting problem (CF) that occurs in continual
learning (CL) settings for a task such as intent recognition. While existing
dialogue systems research has explored replay-based and regularization-based
methods to this end, the effect of domain ordering on the CL performance of
intent recognition models remains unexplored. If understood well, domain
ordering has the potential to be an orthogonal technique that can be leveraged
alongside existing techniques such as experience replay. Our work fills this
gap by comparing the impact of three domain-ordering strategies (min-sum path,
max-sum path, random) on the CL performance of a generative intent recognition
model. Our findings reveal that the min-sum path strategy outperforms the
others in reducing catastrophic forgetting when training on the 220M T5-Base
model. However, this advantage diminishes with the larger 770M T5-Large model.
These results underscores the potential of domain ordering as a complementary
strategy for mitigating catastrophic forgetting in continually learning intent
recognition models, particularly in resource-constrained scenarios.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要