2-Entity Random Sample Consensus for Robust Visual Localization: Framework, Methods, and Verifications

Robust and efficient visual localization is essential for numerous robotic applications. However, it remains a challenging problem especially when significant environmental or perspective changes are present, as there are high percentage of outliers, i.e., incorrect feature matches between the query image and the map. In this article, we propose a novel 2-entity random sample consensus (RANSAC) framework using three-dimensional-two-dimensional point and line feature matches for visual localization with the aid of inertial measurements and derive minimal closed-form solutions using only 1 point 1 line or 2 point matches for both monocular and multi-camera system. The proposed 2-entity RANSAC can achieve higher robustness against outliers as multiple types of features are utilized and the number of matches needed to compute a pose is reduced. Furthermore, we propose a learning-based sampling strategy selection mechanism and a feature scoring network to be adaptive to different environmental characteristics such as structured and unstructured. Finally, both simulation and real-world experiments are performed to validate the robustness and effectiveness of the proposed method in scenarios with long-term and perspective changes.