Cooperative Optimization of Traffic Signals and Vehicle Speed Using a Novel Multi-agent Deep Reinforcement Learning

Using wireless communication and sensor detection technologies, the cooperative vehicle infrastructure system (CVIS) can acquire a wealth of vehicle and road information to provide data support for traffic participants. Deep reinforcement learning (DRL) has been proven to be a promising method for real-time decision-making based on high-dimensional data, which is widely used in traffic control. However, the existing DRL-based research mostly ignores the deep cooperation between the vehicle and the road. In this paper, we present a cooperative optimization of traffic signals and vehicle speed based on multi-agent deep reinforcement learning (COTV-MADRL), aiming to reduce unnecessary stops at the intersection and enhance traffic efficiency. The proposed COTV-MADRL includes two types of agents, called the Light-agent and the Vehicle-agent, which make the policy for traffic lights and vehicles, respectively. To achieve refined control and smoothen traffic flow, the Light-agent adopts a hierarchical architecture to realize the macro-control of the signal cycle and the micro-control of the phase. Meanwhile, the Vehicle-agent also smoothens the traffic flow by harmonizing the speed and the reward design considers the trade-off between efficiency and comfort by referring to human driving behavior. With the support of CVIS, Light-agent and Vehicle-agent can collaborate in the form of information interaction. We conduct experiments on 108 signalized intersections using real online car-hailing data, and the simulation results show that the proposed COTV-MADRL significantly outperforms the conventional methods and several baseline DRL methods.