Distributed reinforcement learning for a traffic engineering application

In this paper, we report on novel reinforcement learning tech- niques applied to a real-world application. The problem do- main, a traffic engineering application, is formulated as a distributed reinforcement learning problem, where the re- turns of many agents are simultaneously updating a single shared policy. Learning occurs off-line in a traffic simulator, which allows us to retrieve and exploit good transient poli- cies even in the presence of instabilities in the learning. We introduce two new algorithms developed for this situation, one which is a value function based, and one that employs a direct policy evaluation approach. While the latter is the- oretically better motivated in several ways than the former, we find both perform comparably well in this domain and for the formulation we use.