Iterative Optical Flow Refinement For High Resolution Images

These days convolutional neural networks (CNNs) are one of the most popular techniques for optical flow estimation yielding exceptional results on many benchmarks. However, the size of their perceptive fields are fixed and displacements not present in the training set cannot be estimated reliably. Additionally, these CNNs need to learn numerous parameters which increases memory consumption. Consequently, their application to high resolution images is impractical, as displacement and memory consumption increase prohibitively. To overcome these limitations, we present an iterative flow refinement approach that can adapt any flow estimation CNN to arbitrary resolution images. In a pyramidal approach, we perform flow estimation for image patches at multiple increasing image resolutions while matching the patches based on the flow of previous iterations. We evaluate our approach using different baselines, displacements and resolutions. The results show that flow estimators can be adapted to high resolution and even panorama images while preserving fine details and reliably handling large displacements without retraining.