Content-adaptive Rate-Distortion Modeling for Frame-level Rate Control in Versatile Video Coding

Rate control (RC) plays an essential role in video coding. RC algorithms based on more accurate rate-distortion (R-D) models often achieve higher control precision and better R-D performance. The most widely used R-D model for HEVC and VVC is the hyperbolic R-D model, which is equivalent to a linear relationship between $\ln {R}$ and $\ln {D}$ . Due to its high accuracy, few studies have attempted to improve the accuracy of the hyperbolic R-D model further. Intuitively, we consider that the accuracy of the hyperbolic R-D model could be further improved by increasing the order of the R-D model. However, this may also increase the number of model parameters to be estimated, which may not benefit the one-pass RC precision. In this paper, we first explicitly note that there is a trade-off between the order of the R-D model and the difficulty in estimating the model parameters in one-pass RC. Then, motivated by the trade-off, we propose high-order R-D models and the corresponding one-pass frame-level RC algorithms for video coding. Finally, we introduce the quadratic R-D model into frame-level RC in the VVC Test Model (VTM-19.2) and provide a content-adaptive model selection between the first-order and second-order R-D models. Experimental results show that the proposed frame-level RC algorithm based on the quadratic R-D model reduces the average frame-level bitrate error by 5.30%, 3.11%, and 13.45% and achieves 0.49%, 0.65%, and 0.35% BD-Rate savings under Low Delay B (LDB), Low Delay P (LDP), and Random Access (RA) configurations, respectively, when compared to the default RC algorithm used in VTM-19.2.