Diffusion Time-step Curriculum for One Image to 3D Generation
CVPR 2024(2024)
摘要
Score distillation sampling (SDS) has been widely adopted to overcome the
absence of unseen views in reconstructing 3D objects from a single
image. It leverages pre-trained 2D diffusion models as teacher to guide the
reconstruction of student 3D models. Despite their remarkable success,
SDS-based methods often encounter geometric artifacts and texture saturation.
We find out the crux is the overlooked indiscriminate treatment of diffusion
time-steps during optimization: it unreasonably treats the student-teacher
knowledge distillation to be equal at all time-steps and thus entangles
coarse-grained and fine-grained modeling. Therefore, we propose the Diffusion
Time-step Curriculum one-image-to-3D pipeline (DTC123), which involves both the
teacher and student models collaborating with the time-step curriculum in a
coarse-to-fine manner. Extensive experiments on NeRF4, RealFusion15, GSO and
Level50 benchmark demonstrate that DTC123 can produce multi-view consistent,
high-quality, and diverse 3D assets. Codes and more generation demos will be
released in https://github.com/yxymessi/DTC123.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要