DOF-GS: Adjustable Depth-of-Field 3D Gaussian Splatting for Refocusing,Defocus Rendering and Blur Removal
CoRR(2024)
摘要
3D Gaussian Splatting-based techniques have recently advanced 3D scene
reconstruction and novel view synthesis, achieving high-quality real-time
rendering. However, these approaches are inherently limited by the underlying
pinhole camera assumption in modeling the images and hence only work for
All-in-Focus (AiF) sharp image inputs. This severely affects their
applicability in real-world scenarios where images often exhibit defocus blur
due to the limited depth-of-field (DOF) of imaging devices. Additionally,
existing 3D Gaussian Splatting (3DGS) methods also do not support rendering of
DOF effects.
To address these challenges, we introduce DOF-GS that allows for rendering
adjustable DOF effects, removing defocus blur as well as refocusing of 3D
scenes, all from multi-view images degraded by defocus blur. To this end, we
re-imagine the traditional Gaussian Splatting pipeline by employing a finite
aperture camera model coupled with explicit, differentiable defocus rendering
guided by the Circle-of-Confusion (CoC). The proposed framework provides for
dynamic adjustment of DOF effects by changing the aperture and focal distance
of the underlying camera model on-demand. It also enables rendering varying DOF
effects of 3D scenes post-optimization, and generating AiF images from
defocused training images. Furthermore, we devise a joint optimization strategy
to further enhance details in the reconstructed scenes by jointly optimizing
rendered defocused and AiF images. Our experimental results indicate that
DOF-GS produces high-quality sharp all-in-focus renderings conditioned on
inputs compromised by defocus blur, with the training process incurring only a
modest increase in GPU memory consumption. We further demonstrate the
applications of the proposed method for adjustable defocus rendering and
refocusing of the 3D scene from input images degraded by defocus blur.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要