Dynamic Reward Adjustment in Multi-Reward Reinforcement Learning for Counselor Reflection Generation
arxiv(2024)
摘要
In this paper, we study the problem of multi-reward reinforcement learning to
jointly optimize for multiple text qualities for natural language generation.
We focus on the task of counselor reflection generation, where we optimize the
generators to simultaneously improve the fluency, coherence, and reflection
quality of generated counselor responses. We introduce two novel bandit
methods, DynaOpt and C-DynaOpt, which rely on the broad strategy of combining
rewards into a single value and optimizing them simultaneously. Specifically,
we employ non-contextual and contextual multi-arm bandits to dynamically adjust
multiple reward weights during training. Through automatic and manual
evaluations, we show that our proposed techniques, DynaOpt and C-DynaOpt,
outperform existing naive and bandit baselines, showcasing their potential for
enhancing language models.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要