OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments
arxiv(2023)
摘要
As a fundamental task of vision-based perception, 3D occupancy prediction
reconstructs 3D structures of surrounding environments. It provides detailed
information for autonomous driving planning and navigation. However, most
existing methods heavily rely on the LiDAR point clouds to generate occupancy
ground truth, which is not available in the vision-based system. In this paper,
we propose an OccNeRF method for training occupancy networks without 3D
supervision. Different from previous works which consider a bounded scene, we
parameterize the reconstructed occupancy fields and reorganize the sampling
strategy to align with the cameras' infinite perceptive range. The neural
rendering is adopted to convert occupancy fields to multi-camera depth maps,
supervised by multi-frame photometric consistency. Moreover, for semantic
occupancy prediction, we design several strategies to polish the prompts and
filter the outputs of a pretrained open-vocabulary 2D segmentation model.
Extensive experiments for both self-supervised depth estimation and 3D
occupancy prediction tasks on nuScenes and SemanticKITTI datasets demonstrate
the effectiveness of our method.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要