Self-supervised Video Object Segmentation with Distillation Learning of Deformable Attention
CoRR(2024)
摘要
Video object segmentation is a fundamental research problem in computer
vision. Recent techniques have often applied attention mechanism to object
representation learning from video sequences. However, due to temporal changes
in the video data, attention maps may not well align with the objects of
interest across video frames, causing accumulated errors in long-term video
processing. In addition, existing techniques have utilised complex
architectures, requiring highly computational complexity and hence limiting the
ability to integrate video object segmentation into low-powered devices. To
address these issues, we propose a new method for self-supervised video object
segmentation based on distillation learning of deformable attention.
Specifically, we devise a lightweight architecture for video object
segmentation that is effectively adapted to temporal changes. This is enabled
by deformable attention mechanism, where the keys and values capturing the
memory of a video sequence in the attention module have flexible locations
updated across frames. The learnt object representations are thus adaptive to
both the spatial and temporal dimensions. We train the proposed architecture in
a self-supervised fashion through a new knowledge distillation paradigm where
deformable attention maps are integrated into the distillation loss. We
qualitatively and quantitatively evaluate our method and compare it with
existing methods on benchmark datasets including DAVIS 2016/2017 and
YouTube-VOS 2018/2019. Experimental results verify the superiority of our
method via its achieved state-of-the-art performance and optimal memory usage.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要