NLBAC: A Neural Ordinary Differential Equations-based Framework for Stable and Safe Reinforcement Learning
CoRR(2024)
摘要
Reinforcement learning (RL) excels in applications such as video games and
robotics, but ensuring safety and stability remains challenging when using RL
to control real-world systems where using model-free algorithms suffering from
low sample efficiency might be prohibitive. This paper first provides safety
and stability definitions for the RL system, and then introduces a Neural
ordinary differential equations-based Lyapunov-Barrier Actor-Critic (NLBAC)
framework that leverages Neural Ordinary Differential Equations (NODEs) to
approximate system dynamics and integrates the Control Barrier Function (CBF)
and Control Lyapunov Function (CLF) frameworks with the actor-critic method to
assist in maintaining the safety and stability for the system. Within this
framework, we employ the augmented Lagrangian method to update the RL-based
controller parameters. Additionally, we introduce an extra backup controller in
situations where CBF constraints for safety and the CLF constraint for
stability cannot be satisfied simultaneously. Simulation results demonstrate
that the framework leads the system to approach the desired state and allows
fewer violations of safety constraints with better sample efficiency compared
to other methods.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要