Texture Grouping and Statistical Optimization Based Mode Prediction Decision Algorithm for Fast HEVC Intra Coding

Abstract HEVC (High Efficiency Video Coding), as one of the newest international video coding standard, can achieve about 50% bit rate reduction compared with H.264/AVC (Advanced Video Coding) at the same perceptual quality due to the use of flexible CTU(coding tree unit) structure, but at the same time, it also dramatically adds the higher computational complexity for HEVC. With the aim of reducing the computational complexity, a texture grouping and statistical optimization based mode prediction decision algorithm is proposed for HEVC intra coding in this paper. The contribution of this paper lies in the fact that we successfully use the texture information grouping and statistical probability optimization technology to rapidly determine the optimal prediction mode for the current PU, which can reduce many unnecessary prediction and calculation operations of HCost (Hadamard Cost) and RDCost (Rate Distortion Cost) in HEVC, thus saving much computation complexity for HEVC. Specially, in our scheme, firstly we group 35 intra prediction modes into 5 subsets of candidate modes list according to its texture information of edge in the current PU, and each subset only contains 11 intra prediction modes, which can greatly reduce many traversing number of candidate mode in RMD (Rough Mode Decision) from 35 to 11 prediction modes; Secondly we use the statistical probability of the first candidate modes in candidate modes list as well as MPM selected as the optimal prediction mode to reduce the number of candidate modes in RDO(Rate Distortion Optimization), which can reduce the number of candidate modes from 3+MPM or 8+MPM to 2 candidate modes; At last, we use the number of candidate modes determined above to quickly find the optimal prediction mode with the minimum RDCost by RDO process. As a result, the computational complexity of HEVC can be efficiently reduced by our proposed scheme. And the simulation results of our experiments show that our proposed intra mode prediction decision algorithm based on texture information grouping and statistical probability optimization in this paper can reduce about 46.13% computational complexity on average only at a cost of 0.67% bit rate increase and 0.056db PSNR decline compared with the standard reference HM16.1 algorithm.