Robust bundle adjustment for large-scale structure from motion

Bundle adjustment (BA) is the problem of refining a visual reconstruction to produce jointly optimal 3D structure and viewing parameter (camera pose and or calibration) estimates, and it is almost always used as the last step of feature-based 3D reconstruction algorithm. Generally, the result of Structure from Motion (SFM) mainly relies on the quality of BA. The problem of BA is often formulated as a nonlinear least squares problem, where the data arises from keypoints matching. For 3D reconstruction, mismatched keypoints may cause serious problems, even a single mismatch will affect the entire reconstruction. Therefore, to further impove the robustness of BA algorithm is very necessary. In this paper, we propose a robust Bundle Adjustment (RBA) algorithm to optimize the initial 3D point-clouds and camera parameters which are produced by the SFM system. In the proposed RBA algorithm, we firstly use the Huber loss function to potentially down-weight outliers. Secondly, we split a large-scale bundle adjustment problem into some small ones by making use of the sparsity between 3D points and the cameras for reducing the requirements of memory. Thirdly, according to the inherent property of the matrix after it spare decompose, we use a fast matrix factorization algorithm to solve the normal equation to avoid calculating the inverse of large-scale matrix. Finally, we evaluate the proposed RBA method and compare it with the state-of-the-art methods on the synthetic dataset, BAL benchmark and real image datasets, respectively. Experimental results show that the proposed RBA method clearly outperforms the state-of-the-art methods on both computational cost and precision.