Towards Online Real-Time Memory-based Video Inpainting Transformers
ICLR 2023(2024)
摘要
Video inpainting tasks have seen significant improvements in recent years
with the rise of deep neural networks and, in particular, vision transformers.
Although these models show promising reconstruction quality and temporal
consistency, they are still unsuitable for live videos, one of the last steps
to make them completely convincing and usable. The main limitations are that
these state-of-the-art models inpaint using the whole video (offline
processing) and show an insufficient frame rate. In our approach, we propose a
framework to adapt existing inpainting transformers to these constraints by
memorizing and refining redundant computations while maintaining a decent
inpainting quality. Using this framework with some of the most recent
inpainting models, we show great online results with a consistent throughput
above 20 frames per second. The code and pretrained models will be made
available upon acceptance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要