SNI-SLAM: Semantic Neural Implicit SLAM
CVPR 2024(2023)
摘要
We propose SNI-SLAM, a semantic SLAM system utilizing neural implicit
representation, that simultaneously performs accurate semantic mapping,
high-quality surface reconstruction, and robust camera tracking. In this
system, we introduce hierarchical semantic representation to allow multi-level
semantic comprehension for top-down structured semantic mapping of the scene.
In addition, to fully utilize the correlation between multiple attributes of
the environment, we integrate appearance, geometry and semantic features
through cross-attention for feature collaboration. This strategy enables a more
multifaceted understanding of the environment, thereby allowing SNI-SLAM to
remain robust even when single attribute is defective. Then, we design an
internal fusion-based decoder to obtain semantic, RGB, Truncated Signed
Distance Field (TSDF) values from multi-level features for accurate decoding.
Furthermore, we propose a feature loss to update the scene representation at
the feature level. Compared with low-level losses such as RGB loss and depth
loss, our feature loss is capable of guiding the network optimization on a
higher-level. Our SNI-SLAM method demonstrates superior performance over all
recent NeRF-based SLAM methods in terms of mapping and tracking accuracy on
Replica and ScanNet datasets, while also showing excellent capabilities in
accurate semantic segmentation and real-time semantic mapping.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要