Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement
2024 International Conference on 3D Vision (3DV)(2024)
Abstract
Visual localization techniques rely upon some underlying scene representation
to localize against. These representations can be explicit such as 3D SFM map
or implicit, such as a neural network that learns to encode the scene. The
former requires sparse feature extractors and matchers to build the scene
representation. The latter might lack geometric grounding not capturing the 3D
structure of the scene well enough. This paper proposes to jointly learn the
scene representation along with a 3D dense feature field and a 2D feature
extractor whose outputs are embedded in the same metric space. Through a
contrastive framework we align this volumetric field with the image-based
extractor and regularize the latter with a ranking loss from learned surface
information. We learn the underlying geometry of the scene with an implicit
field through volumetric rendering and design our feature field to leverage
intermediate geometric information encoded in the implicit field. The resulting
features are discriminative and robust to viewpoint change while maintaining
rich encoded information. Visual localization is then achieved by aligning the
image-based features and the rendered volumetric features. We show the
effectiveness of our approach on real-world scenes, demonstrating that our
approach outperforms prior and concurrent work on leveraging implicit scene
representations for localization.
MoreTranslated text
Key words
Self-supervised Learning,Camera Pose,Implicit Features,Neural Field,Pose Refinement,Neural Network,3D Features,Visual Localization,2D Feature,3D Field,Scene Representation,Implicit Representation,Ranking Loss,Scene Geometry,Concurrent Work,Volumetric Features,Feature Maps,Point Cloud,Latent Space,Neural Representations,Query Image,3D Point,Pose Estimation,Implicit Model,Signed Distance Function,Color Information,Pre-trained Encoder,View Synthesis,Image Database,Image Retrieval
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined