SIF-STGDAN: A Social Interaction Force Spatial-Temporal Graph Dynamic Attention Network for Decision-Making of Connected and Autonomous Vehicles

The collaborative decision-making technology of connected and autonomous vehicles (CAVs) is critical in today’s autonomous driving. Recently, graph reinforcement learning (GRL)-based methods have demonstrated exemplary performance in solving decision-making problems by implementing graphic technologies. However, current GRL-based research faces the challenge of modeling the interaction completely and extracting driving features efficiently. To address these issues, this paper proposes a social interaction force (SIF) spatial-temporal graph dynamic attention network (SIF-STGDAN) to solve the decision-making of CAVs. First, a SIF model is established to better represent the mutual effect between vehicles; an on-ramp merging scenario is then constructed and modeled by graph representation. Then, the SIF-STGDAN is proposed by combining the temporal convolutional network (TCN) and graph dynamic attention network to extract the graphic features of the on-ramp scenario efficiently, and the double deep q-learning (DDQN) algorithm is utilized to generate the optimized driving strategies for CAVs. Finally, experiments are conducted, and results show that our proposed SIF-STGDAN outperforms the baselines in terms of safety, efficiency, and model stability.