A traffic light control method based on multi-agent deep reinforcement learning algorithm

Abstract An efficient way to cope with traffic congestion is Intelligent Traffic Light Control (ITLC). Particularly, multi-agent based intelligent traffic light control algorithms are more popular and efficient than others. But there are still some problems in these algorithms. Firstly, an efficient communication mechanism is needed. As current traffic condition of an intersection can impact other intersections' future traffic condition, every agent should know the traffic condition of other intersections through communication. Thus, a new communication mechanism should be designed. By using this mechanism, traffic condition of an intersection can be passed to several relevant agents easily. And traffic condition of an intersection can be described simply and clearly. Secondly, asynchronization of message should be considered while processing received messages. As duration time of traffic lights' cycles is different from agent to agent and message sending event happens at the end of each traffic light cycle, different agents will send message at different time. Moreover, every agent tries to process the received messages at the end of each traffic light cycle as well. So it is hard for an agent to decide which message is the latest one and the most valuable. Thirdly, a new reward calculating method should be designed. In the traditional reinforcement learning based ITLC algorithms, only queue length of congested cars or waiting time of these cars is considered while calculating reward value. But, both of them are very important for measuring the level of traffic congestion. So a new reward calculation method is needed. To solve all these problems, in this paper, a new ITLC algorithm is proposed. At the same time, a new message sending and processing method is proposed and adopted by this ITLC algorithm. The first and the second problems described above are tried to be solved by using this new method. Besides, to measure traffic congestion in a more reasonable way, a new reward calculation method is proposed and used. This method takes both waiting time and queue length into consideration.