No-Regret Reductions for Imitation Learning and Structured Prediction
Clinical Orthopaedics and Related Research(2011)
摘要
Sequential prediction problems such as imitation learning, where future
observations depend on previous predictions (actions), violate the common
i.i.d. assumptions made in statistical learning. This leads to poor performance
in both theory and often in practice. Some recent approaches provide stronger
performance guarantees in this setting, but remain somewhat unsatisfactory as
they train either non-stationary or a stochastic policies and require a large
number of iterations. In this paper, we propose a new iterative algorithm,
which trains a stationary deterministic policy, that can be seen as a no regret
algorithm in an online learning setting. We show that any such no regret
algorithm, combined with additional reduction assumptions, must find a policy
with good performance under the distribution of observations it induces in such
sequential settings. We additionally show that this new approach outperforms
previous approaches on two challenging imitation learning problem and a
benchmark sequence labeling problem.
更多查看译文
关键词
iterative algorithm
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要