Scale-Relation Joint Decoupling Network for Remote Sensing Image Semantic Segmentation.

As we all know, remote sensing (RS) images contain multiscale and numerous RS objects, along with massive and complex spatial topological relationships, such as the adjacency, proximity relationships of same-scale objects, and inclusion relationships of cross-scale objects. However, the existing semantic segmentation methods have never explored the cross-scale relationships, which are especially important when it comes to the situation that the RS objects cannot be accurately identified, they could be supplemented by the surrounding contents. To address the above concern, we propose a scale-relation joint decoupling network (SRJDN) for the semantic segmentation of RS images by simultaneously considering decoupling scales and decoupling relationships to excavate more complete relationships of multiscale RS objects. The SRJDN is performed by following three steps, namely, scale decoupling (SD), relation decoupling (RD), and fine-granularity guided fusion (FGF). The SD module uses dilated convolution with different rates to decouple RS objects into different scale feature groups, from small to large scales. Afterward, the RD considers all the spatial topological relationships and decouples these relationships according to the scale, which is divided into two parts, including same-scale relation extraction (SSRE) and cross-scale relation extraction (CSRE). The SSRE establishes the graph structures at each scale independently to mine the relationships of same-scale RS objects and the CSRE constructs the graph in a unified pattern between cross-scales to explore cross-scale target relationships. Third, the FGF module regards small-scale features as fine-granularity representation and applies its attention map to guide the learning of other scale features, which could mine more reliable and comprehensive saliency information and improve the feature consistency. Numerical experiments conducted on two large-scale fine-resolution RS image datasets empirically demonstrate the robustness of the proposed joint decoupling strategy and the effectiveness of FGF in RS image semantic segmentation tasks.