UC-NeRF: Neural Radiance Field for Under-Calibrated multi-view cameras in autonomous driving
ICLR 2024(2023)
Abstract
Multi-camera setups find widespread use across various applications, such as
autonomous driving, as they greatly expand sensing capabilities. Despite the
fast development of Neural radiance field (NeRF) techniques and their wide
applications in both indoor and outdoor scenes, applying NeRF to multi-camera
systems remains very challenging. This is primarily due to the inherent
under-calibration issues in multi-camera setup, including inconsistent imaging
effects stemming from separately calibrated image signal processing units in
diverse cameras, and system errors arising from mechanical vibrations during
driving that affect relative camera poses. In this paper, we present UC-NeRF, a
novel method tailored for novel view synthesis in under-calibrated multi-view
camera systems. Firstly, we propose a layer-based color correction to rectify
the color inconsistency in different image regions. Second, we propose virtual
warping to generate more viewpoint-diverse but color-consistent virtual views
for color correction and 3D recovery. Finally, a spatiotemporally constrained
pose refinement is designed for more robust and accurate pose calibration in
multi-camera systems. Our method not only achieves state-of-the-art performance
of novel view synthesis in multi-camera setups, but also effectively
facilitates depth estimation in large-scale outdoor scenes with the synthesized
novel views.
MoreTranslated text
Key words
Neural radiance field,Multi-cameras,under-calibration,color inconsistency
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined