Generalized Optical Flow based Motion Vector Refinement in AV1

Bi-directional optical flow (BDOF) is a coding tool that has been recently adopted into Versatile Video Coding (VVC) standard. In BDOF, gradients of prediction samples are exploited to refine motion vector (MV) per subblock within a prediction block, and thus enhance inter prediction quality in a two-pass framework. In this work, we extend the concept of BDOF to a more general compound prediction framework and integrated this framework into the AV1 codec. In particular, we support MV refinement not only in bi-directional compound prediction, but also in uni-directional prediction, where the two reference blocks are both from the past or both from the future. Furthermore, two reference blocks are allowed to have arbitrary temporal distances to the current block. For implementation on the AV1 codec, five additional inter compound modes have been added, and the proposed method is performed on top of those modes. Experimental results show more than 2.8% BD rate saving on the Google test set.