Regularizing neural radiance fields from sparse rgb-d inputs

This paper aims to improve neural radiance fields (NeRF) from sparse inputs. NeRF achieves photo-realistic renderings when given dense inputs, while its' performance drops dramatically with the decrease of training views' number. Our insight is that the standard volumetric rendering of NeRF is prone to over-fitting due to the lack of overall geometry and local neighborhood information from limited inputs. To address this issue, we propose a global sampling strategy with a geometry regularization utilizing warped images as augmented pseudo-views to encourage geometry consistency across multi-views. In addition, we introduce a local patch sampling scheme with a patch-based regularization for appearance consistency. Furthermore, our method exploits depth information for explicit geometry regularization. The proposed approach outperforms existing baselines on real benchmarks DTU datasets from sparse inputs and achieves the state of art results.