Taming 3DGS: High-Quality Radiance Fields with Limited Resources
CoRR(2024)
Abstract
3D Gaussian Splatting (3DGS) has transformed novel-view synthesis with its
fast, interpretable, and high-fidelity rendering. However, its resource
requirements limit its usability. Especially on constrained devices, training
performance degrades quickly and often cannot complete due to excessive memory
consumption of the model. The method converges with an indefinite number of
Gaussians – many of them redundant – making rendering unnecessarily slow and
preventing its usage in downstream tasks that expect fixed-size inputs. To
address these issues, we tackle the challenges of training and rendering 3DGS
models on a budget. We use a guided, purely constructive densification process
that steers densification toward Gaussians that raise the reconstruction
quality. Model size continuously increases in a controlled manner towards an
exact budget, using score-based densification of Gaussians with training-time
priors that measure their contribution. We further address training speed
obstacles: following a careful analysis of 3DGS' original pipeline, we derive
faster, numerically equivalent solutions for gradient computation and attribute
updates, including an alternative parallelization for efficient
backpropagation. We also propose quality-preserving approximations where
suitable to reduce training time even further. Taken together, these
enhancements yield a robust, scalable solution with reduced training times,
lower compute and memory requirements, and high quality. Our evaluation shows
that in a budgeted setting, we obtain competitive quality metrics with 3DGS
while achieving a 4–5x reduction in both model size and training time. With
more generous budgets, our measured quality surpasses theirs. These advances
open the door for novel-view synthesis in constrained environments, e.g.,
mobile devices.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined