Multiview plus depth video coding with temporal prediction view synthesis

Multiview video plus depths formats use view synthesis to build intermediate views from existing adjacent views at the receiver side. Traditional view synthesis exploits the disparity information to interpolate an intermediate view by taking into account inter-view correlations. However, temporal correlation between different frames of the intermediate view can be used to improve the synthesis. We propose a new coding scheme for 3D-HEVC that allows us to take full advantage of temporal correlations in the intermediate view and improve the existing synthesis from adjacent views. We use optical flow techniques to derive dense motion vector fields from the adjacent views and then warp them at the level of the intermediate view. This allows us to construct multiple temporal predictions of the synthesized frame. A second contribution is an adaptive fusion method that judiciously selects between temporal and inter-view prediction in order to eliminate artifacts associated with each prediction type. The proposed system is compared against the state-of-theart VSRS-1DFast technique used in 3D-HEVC standardization. 3 intermediary views are synthesized. Gains of up to 1.21 dB Bjontegaard Delta PSNR are shown, when evaluated on several standard multiview video test sequences.