Single-image piecewise planar reconstruction of urban buildings based on geometric priors

Single-image 3D urban building reconstruction is an important and challenging topic in computer vision. However, it is a highly ill-posed problem due to the intrinsic geometrical ambiguity in a 3D space. To address the problem, this paper presents a novel single-image piecewise planar reconstruction method based on geometric priors. The proposed method utilizes geometric priors (e.g., plane orientation and intersection angles between planes) learned through convolutional neural networks to handle three challenging subtasks sequentially: (1) detecting planar regions with structure information (e.g., each component edge is associated with two planes intersecting at a specified angle); (2) inferring the planes corresponding to the planar regions progressively by taking full advantage of diverse constraints (e.g., structure and proximity constraints) based on geometric priors; and (3) globally optimizing the resulting planes under a unified framework that incorporates image cues and geometric priors. Experimental results on two public datasets released by CASIA and our own dataset demonstrate that the proposed method significantly outperforms five state-of-the-art methods (including two geometry-based methods and three learning-based methods) by an approximately 20 % margin in terms of accuracy. Code and data are available at https://github.com/three-dimensional-computer-vision/single-image-reconstruction.