Real-Time Model Predictive Control and System Identification Using Differentiable Simulation

Transferring a controller from a simulated environment to a physical system is regarded as a challenging problem in robotics. We present a method for continuous improvement of modeling and control after deploying the robot to a dynamically-changing target environment. We develop a differentiable physics simulation framework that simultaneously performs online system identification and optimal control using the incoming observations from the target environment in real time. To ensure robust system identification against noisy observations, we devise an algorithm to assess the confidence of our estimated parameters using numerical analysis of the dynamic equations. To ensure real-time optimal control, we adapt start time of the optimization window so that the optimized actions can be replenished ahead of consumption, while staying as up-to-date with new information as possible. The constantly re-planning based on a constantly improving model allows the robot to swiftly adapt to the changing environment using real-world data in a sample-efficient way. Thanks to a fast differentiable physics simulator, both system identification and control can be solved efficiently in real time. We demonstrate our method on a set of examples in simulation and on a real robot. Our method can outperform all baseline methods in different experiments.