Depth-guided NeRF Training via Earth Mover's Distance
CoRR(2024)
Abstract
Neural Radiance Fields (NeRFs) are trained to minimize the rendering loss of
predicted viewpoints. However, the photometric loss often does not provide
enough information to disambiguate between different possible geometries
yielding the same image. Previous work has thus incorporated depth supervision
during NeRF training, leveraging dense predictions from pre-trained depth
networks as pseudo-ground truth. While these depth priors are assumed to be
perfect once filtered for noise, in practice, their accuracy is more
challenging to capture. This work proposes a novel approach to uncertainty in
depth priors for NeRF supervision. Instead of using custom-trained depth or
uncertainty priors, we use off-the-shelf pretrained diffusion models to predict
depth and capture uncertainty during the denoising process. Because we know
that depth priors are prone to errors, we propose to supervise the ray
termination distance distribution with Earth Mover's Distance instead of
enforcing the rendered depth to replicate the depth prior exactly through
L2-loss. Our depth-guided NeRF outperforms all baselines on standard depth
metrics by a large margin while maintaining performance on photometric
measures.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined