CromSS: Cross-modal pre-training with noisy labels for remote sensing image segmentation
CoRR(2024)
Abstract
We study the potential of noisy labels y to pretrain semantic segmentation
models in a multi-modal learning framework for geospatial applications.
Specifically, we propose a novel Cross-modal Sample Selection method (CromSS)
that utilizes the class distributions P^(d)(x,c) over pixels x and classes c
modelled by multiple sensors/modalities d of a given geospatial scene.
Consistency of predictions across sensors d is jointly informed by the
entropy of P^(d)(x,c). Noisy label sampling we determine by the confidence of
each sensor d in the noisy class label, P^(d)(x,c=y(x)). To verify the
performance of our approach, we conduct experiments with Sentinel-1 (radar) and
Sentinel-2 (optical) satellite imagery from the globally-sampled SSL4EO-S12
dataset. We pair those scenes with 9-class noisy labels sourced from the Google
Dynamic World project for pretraining. Transfer learning evaluations
(downstream task) on the DFC2020 dataset confirm the effectiveness of the
proposed method for remote sensing image segmentation.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined