Scanline-based fast algorithm and pipelined hardware design of rate-distortion optimized quantization for AVS3

The third generation AVS video coding standard (AVS3) is developed by the China AVS working group, and it achieves a 50% bitrate saving compared to the previous AVS2 standard by adopting several efficient coding tools. Ratedistortion optimized quantization (RDOQ) is one of the effective tools, which decides the optimal quantized coefficient level based on rate-distortion cost. The huge computation complexity and strong data dependency in RDOQ impede the real-time applications, where there is no hardware video encoder supporting RDOQ. To resolve this problem, we propose a zigzag scanline-based fast algorithm and pipelined hardware design for AVS3 RDOQ. Firstly, we introduce an approach to decrease computation and then propose a fully parallel coefficient level decision method where the context is correlated to the current position. Secondly, a zig-zag scanline-based algorithm is proposed to decide the last non-zero pixel position, where the scanlines can be paralleled. Finally, its corresponding hardware architecture design is proposed. Experimental results show that the proposed hardware architecture can support a throughput of 4K@30fps with negligible performance loss when working at 200MHz.