Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume
CoRR(2023)
Abstract
The full 4D cost volume in Recurrent All-Pairs Field Transforms (RAFT) or
global matching by Transformer achieves impressive performance for optical flow
estimation. However, their memory consumption increases quadratically with
input resolution, rendering them impractical for high-resolution images. In
this paper, we present MeFlow, a novel memory-efficient method for
high-resolution optical flow estimation. The key of MeFlow is a recurrent local
orthogonal cost volume representation, which decomposes the 2D search space
dynamically into two 1D orthogonal spaces, enabling our method to scale
effectively to very high-resolution inputs. To preserve essential information
in the orthogonal space, we utilize self attention to propagate feature
information from the 2D space to the orthogonal space. We further propose a
radius-distribution multi-scale lookup strategy to model the correspondences of
large displacements at a negligible cost. We verify the efficiency and
effectiveness of our method on the challenging Sintel and KITTI benchmarks, and
real-world 4K ($2160\!\times\!3840$) images. Our method achieves competitive
performance on both Sintel and KITTI benchmarks, while maintaining the highest
memory efficiency on high-resolution inputs.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined