OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learning
arxiv(2024)
摘要
Offline safe reinforcement learning (RL) aims to train a policy that
satisfies constraints using a pre-collected dataset. Most current methods
struggle with the mismatch between imperfect demonstrations and the desired
safe and rewarding performance. In this paper, we introduce OASIS (cOnditionAl
diStributIon Shaping), a new paradigm in offline safe RL designed to overcome
these critical limitations. OASIS utilizes a conditional diffusion model to
synthesize offline datasets, thus shaping the data distribution toward a
beneficial target domain. Our approach makes compliance with safety constraints
through effective data utilization and regularization techniques to benefit
offline safe RL training. Comprehensive evaluations on public benchmarks and
varying datasets showcase OASIS's superiority in benefiting offline safe RL
agents to achieve high-reward behavior while satisfying the safety constraints,
outperforming established baselines. Furthermore, OASIS exhibits high data
efficiency and robustness, making it suitable for real-world applications,
particularly in tasks where safety is imperative and high-quality
demonstrations are scarce.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要