Autonomous Driving Policy Continual Learning With One-Shot Disengagement Case

Disengagement cases during naturalistic driving are rare or even one-shot, but valuable for autonomous driving. The autonomous vehicles are necessary to continually learn from these disengagement cases, to improve the policy for better performance when next time meeting these cases. Manually adjusting the policy or adding the rules to fix these disengagement cases may cause engineering burden and may contradict other driving functions. To this end, this work proposes a continually learning agent which can automatically get improved once encountering a disengagement case. The main idea is to establish a disengagement-imagination environment, and then train the policy using imagination data for performance improvement, named disengagement-case imagination augmented continual learning (DICL). In the imagination environment, the surrounding objects are designed to first follow the recorded trajectory, and then switch to the interactive models for the policy training. The switch point is carefully designed to make the imagination contain the disengagement reasons but avoid overfitting the collected driving case. This method is evaluated by the real autonomous driving disengagement data, collected from an open-road-testing autonomous vehicle. The results show that the DICL agent can automatically learn to handle the emerging disengagement case and similar cases. This work provides a possible way to make the AV agents automatically get improvement during road testing.