CAT: a coarse-to-fine attention tree for semantic change detection

Semantic change detection (SCD) and land cover mapping (LCM) are always treated as a dual task in the field of remote sensing. However, due to diverse real-world scenarios, many SCD categories are not easy to be clearly recognized, such as “water-vegetation” and “water-tree”, which can be regarded as fine-grained differences. In addition, even a single LCM category is usually difficult to define. For instance, some “vegetation” categories with litter vegetation coverage are easily confused with the general “ground” category. SCD/LCM becomes challenging under both challenges of its fine-grained nature and label ambiguity. In this paper, we tackle the SCD and LCM tasks simultaneously by proposing a coarse-to-fine attention tree (CAT) model. Specifically, it consists of an encoder, a decoder and a coarse-to-fine attention tree module. The encoder-decoder structure extracts the high-level features from input multi-temporal images first and then reconstructs them to return SCD and LCM predictions. Our coarse-to-fine attention tree, on the one hand, utilizes the tree structure to better model a hierarchy of categories by predicting the coarse-grained labels first and then predicting the fine-grained labels later. On the other hand, it applies the attention mechanism to capture discriminative pixel regions. Furthermore, to address label ambiguity in SCD/LCM, we also equip a label distribution learning loss upon our model. Experiments on the large-scale SECOND dataset justify that the proposed CAT model outperforms state-of-the-art models. Moreover, various ablation studies have demonstrated the effectiveness of tailored designs in the CAT model for solving semantic change detection problems.