Collaborative Position Reasoning Network for Referring Image Segmentation
CoRR(2024)
摘要
Given an image and a natural language expression as input, the goal of
referring image segmentation is to segment the foreground masks of the entities
referred by the expression. Existing methods mainly focus on interactive
learning between vision and language to enhance the multi-modal representations
for global context reasoning. However, predicting directly in pixel-level space
can lead to collapsed positioning and poor segmentation results. Its main
challenge lies in how to explicitly model entity localization, especially for
non-salient entities. In this paper, we tackle this problem by executing a
Collaborative Position Reasoning Network (CPRN) via the proposed novel
Row-and-Column interactive (RoCo) and Guided Holistic interactive (Holi)
modules. Specifically, RoCo aggregates the visual features into the row- and
column-wise features corresponding two directional axes respectively. It offers
a fine-grained matching behavior that perceives the associations between the
linguistic features and two decoupled visual features to perform position
reasoning over a hierarchical space. Holi integrates features of the two
modalities by a cross-modal attention mechanism, which suppresses the
irrelevant redundancy under the guide of positioning information from RoCo.
Thus, with the incorporation of RoCo and Holi modules, CPRN captures the visual
details of position reasoning so that the model can achieve more accurate
segmentation. To our knowledge, this is the first work that explicitly focuses
on position reasoning modeling. We also validate the proposed method on three
evaluation datasets. It consistently outperforms existing state-of-the-art
methods.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要