Modeling coupled driving behavior during lane change: A multi-agent Transformer reinforcement learning approach

In a lane change (LC) scenario, the lane change vehicle interacts with surrounding vehicles. The interactions not only affect their driving behaviors but also influence the traffic flow. This study aims to model the coupled behavior of the lane changer and the follower in the target lane during LC. Large-scale real-world connected vehicle (CV) data from the Safety Pilot Model Deployment (SPMD) program are used to extract LCs and study vehicle interactions. A multi-agent Transformer-based deep deterministic policy gradient (MA-TDDPG) method is proposed to model the coupled behaviors during LC. The multi-agent framework can handle the multiple agents’ behaviors with interactions, and the Transformer can process the observation-action memory accurately and efficiently. The MA-TDDPG algorithm can learn the sequential decision-making process over continuous action space during LC with the accommodation of the multi-vehicle interaction and the driver’s memory effect. Compared to traditional supervised learning and reinforcement learning methods, it demonstrates superior performance in imitating the longitudinal and lateral actions of the lane changer and the follower. The findings of this study provide insights into the development of microscopic simulations by producing realistic LC behaviors, and assistance/automation LC systems by generating LC motions and responses conforming to human driving habits. The model also creates an interactive simulation environment and lays the foundation for optimizing driving strategies.