A Fine-Grained Unsupervised Domain Adaptation Framework for Semantic Segmentation of Remote Sensing Images

Unsupervised domain adaptation (UDA) aims at adapting a model from the source domain to the target domain by tackling the issue of domain shift. Cross-domain segmentation of remote sensing images (RSIs) remains a big challenge due to the unique properties of RSIs. On the one hand, the divergence of data distribution in different local regions leads to negative transfer by directly applying the global alignment method in RSIs. On the other hand, the underlying category-level structure in the target domain is often ignored, which confuses the decision of semantic boundaries on the dispersed category features caused by large intraclass variance and small interclass variance in RSIs. In this study, we propose a novel fine-grained adaptation framework combining two stages of global-local alignment and category-level alignment to solve the above-mentioned problems. In the first stage of global-local adaptation, an attention map is derived from an intermediate discriminator and focuses on hard-to-align regions to mitigate negative transfer due to global adversarial learning. In the second stage of category-level adaptation, the category feature compact module is utilized to address the issue of dispersed features in the target domain attained by the cross-domain network, which will facilitate the fine-grained alignment of categories. Experiments under various scenarios, including geographic location variation and spectral band composition variation, demonstrate that the local adaptation and category-level adaptation of RSIs are complementary in the cross-domain segmentation, and the integrated framework helps achieve outstanding performance for UDA semantic segmentation of RSIs.