TSANet: Forecasting Traffic Congestion Patterns from Aerial Videos Using Graphs and Transformers

Forecasting traffic congestion patterns in lane-less traffic scenarios is a complex task because of the combination of high & irregular vehicle densities, fluctuating speeds, and the presence of environmental obstacles. Existing techniques like vehicle counting and density prediction, which successfully estimate congestion in lane-based traffic, are unsuitable for lane-less traffic scenarios due to the irregular and unpredictable nature of traffic density patterns. To overcome these challenges, we propose traffic states to measure congestion patterns in lane-less traffic scenarios. Each traffic state is characterized by the spatio-temporal distribution of neighbouring road users, including vehicles and motorcyclists. We employ traffic graphs to capture the spatial distribution of neighbouring road users. Also, we propose a novel method for the automated construction of traffic graphs by leveraging the detection and tracking of individual road users in aerial videos. Further, in order to incorporate the temporal distribution, we utilize a transformer model to capture the evolution of spatial traffic graphs over time. This enables us to forecast future spatio-temporal distributions and their associated traffic states. Our proposed model, named Traffic State Anticipation Network (TSANet), can effectively forecast future traffic states by analysing sequences of current traffic graphs, thereby enhancing our understanding of evolving traffic patterns in lane-less scenarios. Also, to address the lack of publicly available lane-less traffic datasets, we introduce EyeonTraffic (EoT), a large-scale lane-less traffic dataset containing three hours of aerial videos captured at three busy intersections in Ahmedabad city, India. Experimental results on the EoT dataset demonstrate the efficacy of our proposed TSANet in effectively anticipating traffic states across diverse spatial regions within an intersection. In addition, we also show that TSANet generalizes well for previously unseen intersections, making it suitable for analysing various traffic scenarios without the need for explicit training, thereby enhancing its practical applicability.