Neural Video Compression with Feature Modulation
CoRR(2024)
摘要
The emerging conditional coding-based neural video codec (NVC) shows
superiority over commonly-used residual coding-based codec and the latest NVC
already claims to outperform the best traditional codec. However, there still
exist critical problems blocking the practicality of NVC. In this paper, we
propose a powerful conditional coding-based NVC that solves two critical
problems via feature modulation. The first is how to support a wide quality
range in a single model. Previous NVC with this capability only supports about
3.8 dB PSNR range on average. To tackle this limitation, we modulate the latent
feature of the current frame via the learnable quantization scaler. During the
training, we specially design the uniform quantization parameter sampling
mechanism to improve the harmonization of encoding and quantization. This
results in a better learning of the quantization scaler and helps our NVC
support about 11.4 dB PSNR range. The second is how to make NVC still work
under a long prediction chain. We expose that the previous SOTA NVC has an
obvious quality degradation problem when using a large intra-period setting. To
this end, we propose modulating the temporal feature with a periodically
refreshing mechanism to boost the quality.
problems, we also design a single model that can support both RGB and YUV
colorspaces. Notably, under single intra-frame setting, our codec can achieve
29.7% bitrate saving over previous SOTA NVC with 16% MACs reduction. Our
codec serves as a notable landmark in the journey of NVC evolution. The codes
are at https://github.com/microsoft/DCVC.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要