FSViewFusion: Few-Shots View Generation of Novel Objects
arxiv(2024)
摘要
Novel view synthesis has observed tremendous developments since the arrival
of NeRFs. However, Nerf models overfit on a single scene, lacking
generalization to out of distribution objects. Recently, diffusion models have
exhibited remarkable performance on introducing generalization in view
synthesis. Inspired by these advancements, we explore the capabilities of a
pretrained stable diffusion model for view synthesis without explicit 3D
priors. Specifically, we base our method on a personalized text to image model,
Dreambooth, given its strong ability to adapt to specific novel objects with a
few shots. Our research reveals two interesting findings. First, we observe
that Dreambooth can learn the high level concept of a view, compared to
arguably more complex strategies which involve finetuning diffusions on large
amounts of multi-view data. Second, we establish that the concept of a view can
be disentangled and transferred to a novel object irrespective of the original
object's identify from which the views are learnt. Motivated by this, we
introduce a learning strategy, FSViewFusion, which inherits a specific view
through only one image sample of a single scene, and transfers the knowledge to
a novel object, learnt from few shots, using low rank adapters. Through
extensive experiments we demonstrate that our method, albeit simple, is
efficient in generating reliable view samples for in the wild images. Code and
models will be released.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要