Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agents
CoRR(2024)
摘要
Reinforcement Learning (RL) has made significant strides in enabling
artificial agents to learn diverse behaviors. However, learning an effective
policy often requires a large number of environment interactions. To mitigate
sample complexity issues, recent approaches have used high-level task
specifications, such as Linear Temporal Logic (LTL_f) formulas or Reward
Machines (RM), to guide the learning progress of the agent. In this work, we
propose a novel approach, called Logical Specifications-guided Dynamic Task
Sampling (LSTS), that learns a set of RL policies to guide an agent from an
initial state to a goal state based on a high-level task specification, while
minimizing the number of environmental interactions. Unlike previous work, LSTS
does not assume information about the environment dynamics or the Reward
Machine, and dynamically samples promising tasks that lead to successful goal
policies. We evaluate LSTS on a gridworld and show that it achieves improved
time-to-threshold performance on complex sequential decision-making problems
compared to state-of-the-art RM and Automaton-guided RL baselines, such as
Q-Learning for Reward Machines and Compositional RL from logical Specifications
(DIRL). Moreover, we demonstrate that our method outperforms RM and
Automaton-guided RL baselines in terms of sample-efficiency, both in a
partially observable robotic task and in a continuous control robotic
manipulation task.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要