CoSSegGaussians: Compact and Swift Scene Segmenting 3D Gaussians with Dual Feature Fusion
CoRR(2024)
摘要
We propose Compact and Swift Segmenting 3D Gaussians(CoSSegGaussians), a
method for compact 3D-consistent scene segmentation at fast rendering speed
with only RGB images input. Previous NeRF-based segmentation methods have
relied on time-consuming neural scene optimization. While recent 3D Gaussian
Splatting has notably improved speed, existing Gaussian-based segmentation
methods struggle to produce compact masks, especially in zero-shot
segmentation. This issue probably stems from their straightforward assignment
of learnable parameters to each Gaussian, resulting in a lack of robustness
against cross-view inconsistent 2D machine-generated labels. Our method aims to
address this problem by employing Dual Feature Fusion Network as Gaussians'
segmentation field. Specifically, we first optimize 3D Gaussians under RGB
supervision. After Gaussian Locating, DINO features extracted from images are
applied through explicit unprojection, which are further incorporated with
spatial features from the efficient point cloud processing network. Feature
aggregation is utilized to fuse them in a global-to-local strategy for compact
segmentation features. Experimental results show that our model outperforms
baselines on both semantic and panoptic zero-shot segmentation task, meanwhile
consumes less than 10% inference time compared to NeRF-based methods. Code and
more results will be available at https://David-Dou.github.io/CoSSegGaussians.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要