Highly Detailed and Temporal Consistent Video Stylization via Synchronized Multi-Frame Diffusion.
CoRR(2023)
摘要
Text-guided video-to-video stylization transforms the visual appearance of a
source video to a different appearance guided on textual prompts. Existing
text-guided image diffusion models can be extended for stylized video
synthesis. However, they struggle to generate videos with both highly detailed
appearance and temporal consistency. In this paper, we propose a synchronized
multi-frame diffusion framework to maintain both the visual details and the
temporal consistency. Frames are denoised in a synchronous fashion, and more
importantly, information of different frames is shared since the beginning of
the denoising process. Such information sharing ensures that a consensus, in
terms of the overall structure and color distribution, among frames can be
reached in the early stage of the denoising process before it is too late. The
optical flow from the original video serves as the connection, and hence the
venue for information sharing, among frames. We demonstrate the effectiveness
of our method in generating high-quality and diverse results in extensive
experiments. Our method shows superior qualitative and quantitative results
compared to state-of-the-art video editing methods.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要