CT-MVSNet: Curvature-guided multi-view stereo with transformers

Multi-view stereo (MVS) can fulfill dense three-dimensional reconstruction from a collection of multi-view images. Although deep learning-based MVS has significantly enhanced the reconstruction performance, the reconstruction accuracy and completeness still require improvements to meet the need of real applications of three-dimensional content generation. So, a new MVS method, the curvature-guided multi-view stereo with transformers, is presented. By exploring inter-view relationships and measuring the size of the receptive field and feature information on the image surface using the surface curvature, the proposed method adapts to various candidate scales of curvatures to extract more detailed features adaptively for precise cost computation. Furthermore, a transformer-based feature-matching network is proposed to identify inter-view similarity better and enhance feature-matching accuracy. Additionally, a similarity measurement module based on feature matching integrates curvature and inter-view similarity measurement tightly to further improve reconstruction accuracy. Experiments on the DTU dataset and Tanks and Temples dataset validate the proposed method.