DoubleTake: Geometry Guided Depth Estimation
arxiv(2024)
Abstract
Estimating depth from a sequence of posed RGB images is a fundamental
computer vision task, with applications in augmented reality, path planning
etc. Prior work typically makes use of previous frames in a multi view stereo
framework, relying on matching textures in a local neighborhood. In contrast,
our model leverages historical predictions by giving the latest 3D geometry
data as an extra input to our network. This self-generated geometric hint can
encode information from areas of the scene not covered by the keyframes and it
is more regularized when compared to individual predicted depth maps for
previous frames. We introduce a Hint MLP which combines cost volume features
with a hint of the prior geometry, rendered as a depth map from the current
camera location, together with a measure of the confidence in the prior
geometry. We demonstrate that our method, which can run at interactive speeds,
achieves state-of-the-art estimates of depth and 3D scene reconstruction in
both offline and incremental evaluation scenarios.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined