LiDAR Place Recognition Evaluation with the Oxford Radar RobotCar Dataset Revised.

The Oxford Radar RobotCar dataset has recently become popular in evaluating LiDAR-based methods for place recognition. The Radar dataset is preferred over the original Oxford RobotCar dataset since it has better LiDAR sensors and location ground truth is available for all sequences. However, it turns out that the Radar dataset has serious issues with its ground truth and therefore experimental findings with this dataset can be misleading. We demonstrate how easily this can happen, by varying only the gallery sequence and keeping the training and test sequences fixed. Results of this experiment strongly indicate that the gallery selection is an important consideration for place recognition. However, the finding is a mistake and the difference between galleries can be explained by systematic errors in the ground truth. In this work, we propose a revised benchmark for LiDAR-based place recognition with the Oxford Radar RobotCar dataset. The benchmark includes fixed gallery, training and test sequences, corrected ground truth, and a strong baseline method. All data and code will be made publicly available to facilitate fair method comparison and development.