SOAT-UNET: a transformer-based Siamese over-attention network for change detection

The transformer plays a crucial role in building change detection (BCD) systems, which are important for observing urban development and post-disaster assessment. However, existing technologies often lack the ability to simultaneously attend to object features in bitemporal images and are not sensitive to changes in small target buildings. To address these issues, we propose SOAT-UNet, a novel transformer-based Siamese network with a multi-head over-attention block for CD tasks. Leveraging token-based space, our model extracts long-range contextual relationships and improves feature extraction for small targets. Inspired by human behavior, we generate queries (Q) from two image sets and calculate keys (K) and values (V) from another set, prioritizing regions likely to change. Experimental results demonstrate that our SOAT-UNet achieves superior CD performance compared to previous models on two existing datasets.