Fine-grained Background Representation for Weakly Supervised Semantic Segmentation
IEEE Transactions on Circuits and Systems for Video Technology(2024)
摘要
Generating reliable pseudo masks from image-level labels is challenging in
the weakly supervised semantic segmentation (WSSS) task due to the lack of
spatial information. Prevalent class activation map (CAM)-based solutions are
challenged to discriminate the foreground (FG) objects from the suspicious
background (BG) pixels (a.k.a. co-occurring) and learn the integral object
regions. This paper proposes a simple fine-grained background representation
(FBR) method to discover and represent diverse BG semantics and address the
co-occurring problems. We abandon using the class prototype or pixel-level
features for BG representation. Instead, we develop a novel primitive, negative
region of interest (NROI), to capture the fine-grained BG semantic information
and conduct the pixel-to-NROI contrast to distinguish the confusing BG pixels.
We also present an active sampling strategy to mine the FG negatives
on-the-fly, enabling efficient pixel-to-pixel intra-foreground contrastive
learning to activate the entire object region. Thanks to the simplicity of
design and convenience in use, our proposed method can be seamlessly plugged
into various models, yielding new state-of-the-art results under various WSSS
settings across benchmarks. Leveraging solely image-level (I) labels as
supervision, our method achieves 73.2 mIoU and 45.6 mIoU segmentation results
on Pascal Voc and MS COCO test sets, respectively. Furthermore, by
incorporating saliency maps as an additional supervision signal (I+S), we
attain 74.9 mIoU on Pascal Voc test set. Concurrently, our FBR approach
demonstrates meaningful performance gains in weakly-supervised instance
segmentation (WSIS) tasks, showcasing its robustness and strong generalization
capabilities across diverse domains.
更多查看译文
关键词
Contrastive learning,Fine-grained background representation,Weakly supervised image segmentation
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要