Semantic Category Balance-Aware Involved Anti-Interference Network for Remote Sensing Semantic Segmentation

In recent years, semantic segmentation technology plays an important role in land resource management tasks. However, many classic semantic segmentation methods often fail to obtain satisfactory results for remote sensing images with a large amount of interference information. To improve this situation, we propose semantic category balance-aware involved anti-interference network (SCBANet). SCBANet has an encoder–decoder structure similar to DeeplabV3+. On this basis, we propose clustering-guided semantic decoupling module (CGSDM), consistency-based anti-interference feature extraction module (CAFEM), relevance-based anti-interference feature extraction module (RAFEM), and optional decoder module based on semantic category balance (ODMSCB) to improve the accuracy of semantic segmentation. CGSDM aims to obtain the information of different semantic categories through $K$ - means clustering algorithm. CAFEM performs an average operation on the feature vectors in each semantic category to obtain semantic consistency information. RAFEM deeply excavates the information contained in each semantic category through the modeling method with self-attention mechanism as the core, making the relationship between pixels within each semantic category to be better understood by the model. ODMSCB classifies the feature map according to the balance of different semantic categories, so that different decoders can be applied to feature maps with different semantic category balance. These four parts complement each other, greatly improving the model’s anti-interference ability while also enhancing the ability to handle category imbalance issue. We compared our method with several of the most advanced deep learning methods on the Vaihingen and Potsdam datasets. The final results demonstrate the superiority of our method.