A CU-level adaptive decision method for CNN-based in-loop filtering

Convolutional Neural Network (CNN) has been introduced to in-loop filtering in video coding for further performance improvement. For intra frame coding, a CNN model can be directly trained through learning the correlation between the reconstructed and the original frames, and the obtained model can then be applied to every single reconstructed frame to help improve the video quality. In contrast, for inter frame coding, intertwined reference dependency exists across frames. If a similar procedure of model training and deployment is adopted for inter as that for intra coding, over-smoothed reconstructed frames may be generated, which may further seriously deteriorate the overall coding performance. To address such an issue, state-of-the-art work resorts to the Rate Distortion Optimization (RDO) to determine whether to adopt the conventional or the CNN-based scheme for in-loop filtering, however leading to high computational complexity. In this paper, we propose a Coding-unit (CU) level Adaptive Decision approach (CAD) which employs an early decision for each CU, based on their coding parameters. Experimental results show that the proposed scheme achieves comparable performance with that of the RDO scheme while effectively reduces the encoding time complexity.