Numerical Weather Forecasting using Convolutional-LSTM with Attention and Context Matcher Mechanisms

Numerical weather forecasting with high-resolution physical models requires extensive computational resources on supercomputers, often making it impractical for real-life applications. Alternatively, deep learning methods can provide results within minutes of receiving data. Although baseline deep learning models can make accurate short-term predictions, their performance deteriorates rapidly as the output sequence length increases. However, many real-life scenarios require long-term prediction of certain weather features to mitigate and take advantage of the effects of high-impact weather events. In response, we introduce the Weather Model, which provides rapid and accurate long-term spatial predictions for high-resolution spatio-temporal weather data. We integrate a stacked ConvLSTM network as our building block, given its accuracy in capturing spatial data patterns through convolution operations. Furthermore, we additionally incorporate attention and context matcher mechanisms. The attention mechanism allows the effective usage of the side-information vector by selectively focusing on different parts of the input sequence at each time step. Concurrently, the context matcher mechanism enhances the network’s ability to preserve long-term dependencies. Our Weather Model achieves significant performance improvements compared to baseline deep learning models, including ConvLSTM, TrajGRU and U-Net. Our experimental evaluation involves high-scale, real-world benchmark numerical weather datasets, namely the ERA5 hourly dataset on pressure levels and WeatherBench. Our results demonstrate substantial improvements in identifying spatial and temporal correlations, with attention matrices focusing on distinct parts of the input series to model atmospheric circulations. We also compare our model with high-resolution physical models using benchmark metrics to confirm our Weather Model’s accuracy and interpretability.