Efficient Video Object Segmentation via Modulated Cross-Attention Memory
arxiv(2024)
摘要
Recently, transformer-based approaches have shown promising results for
semi-supervised video object segmentation. However, these approaches typically
struggle on long videos due to increased GPU memory demands, as they frequently
expand the memory bank every few frames. We propose a transformer-based
approach, named MAVOS, that introduces an optimized and dynamic long-term
modulated cross-attention (MCA) memory to model temporal smoothness without
requiring frequent memory expansion. The proposed MCA effectively encodes both
local and global features at various levels of granularity while efficiently
maintaining consistent speed regardless of the video length. Extensive
experiments on multiple benchmarks, LVOS, Long-Time Video, and DAVIS 2017,
demonstrate the effectiveness of our proposed contributions leading to
real-time inference and markedly reduced memory demands without any degradation
in segmentation accuracy on long videos. Compared to the best existing
transformer-based approach, our MAVOS increases the speed by 7.6x, while
significantly reducing the GPU memory by 87
performance on short and long video datasets. Notably on the LVOS dataset, our
MAVOS achieves a J F score of 63.3
(FPS) on a single V100 GPU. Our code and models will be publicly available at:
https://github.com/Amshaker/MAVOS.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要