Signatures Meet Dynamic Programming: Generalizing Bellman Equations for Trajectory Following
CoRR(2023)
摘要
Path signatures have been proposed as a powerful representation of paths that
efficiently captures the path's analytic and geometric characteristics, having
useful algebraic properties including fast concatenation of paths through
tensor products. Signatures have recently been widely adopted in machine
learning problems for time series analysis. In this work we establish
connections between value functions typically used in optimal control and
intriguing properties of path signatures. These connections motivate our novel
control framework with signature transforms that efficiently generalizes the
Bellman equation to the space of trajectories. We analyze the properties and
advantages of the framework, termed signature control. In particular, we
demonstrate that (i) it can naturally deal with varying/adaptive time steps;
(ii) it propagates higher-level information more efficiently than value
function updates; (iii) it is robust to dynamical system misspecification over
long rollouts. As a specific case of our framework, we devise a model
predictive control method for path tracking. This method generalizes integral
control, being suitable for problems with unknown disturbances. The proposed
algorithms are tested in simulation, with differentiable physics models
including typical control and robotics tasks such as point-mass, curve
following for an ant model, and a robotic manipulator.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要