NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation
arxiv(2024)
摘要
The emergence of Neural Radiance Fields (NeRF) has greatly impacted 3D scene
modeling and novel-view synthesis. As a kind of visual media for 3D scene
representation, compression with high rate-distortion performance is an eternal
target. Motivated by advances in neural compression and neural field
representation, we propose NeRFCodec, an end-to-end NeRF compression framework
that integrates non-linear transform, quantization, and entropy coding for
memory-efficient scene representation. Since training a non-linear transform
directly on a large scale of NeRF feature planes is impractical, we discover
that pre-trained neural 2D image codec can be utilized for compressing the
features when adding content-specific parameters. Specifically, we reuse neural
2D image codec but modify its encoder and decoder heads, while keeping the
other parts of the pre-trained decoder frozen. This allows us to train the full
pipeline via supervision of rendering loss and entropy loss, yielding the
rate-distortion balance by updating the content-specific parameters. At test
time, the bitstreams containing latent code, feature decoder head, and other
side information are transmitted for communication. Experimental results
demonstrate our method outperforms existing NeRF compression methods, enabling
high-quality novel view synthesis with a memory budget of 0.5 MB.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要