Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedback
arxiv(2024)
摘要
In many real-world applications, it is hard to provide a reward signal in
each step of a Reinforcement Learning (RL) process and more natural to give
feedback when an episode ends. To this end, we study the recently proposed
model of RL with Aggregate Bandit Feedback (RL-ABF), where the agent only
observes the sum of rewards at the end of an episode instead of each reward
individually. Prior work studied RL-ABF only in tabular settings, where the
number of states is assumed to be small. In this paper, we extend ABF to linear
function approximation and develop two efficient algorithms with near-optimal
regret guarantees: a value-based optimistic algorithm built on a new
randomization technique with a Q-functions ensemble, and a policy optimization
algorithm that uses a novel hedging scheme over the ensemble.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要