CARLA Simulated Data for Rare Road Object Detection

Labeled data are paramount for modern, deep learning object detection models. However, such data are not always available, either due to time and financial constraints or due to the rarity of certain objects. In this paper, we show that the CARLA simulator can be used effectively to provide automatic annotations for custom street-view objects, boosting datasets for objects with few labels. We evaluate our models on real world images and show that low-shot training data expanded by synthetic images rendered in CARLA can provide better performance than training models with low-shot examples alone. To overcome the sim-to-real domain gap, we perform domain randomization by taking advantage of CARLA's diverse simulations of weather conditions, actors, and maps. We train detectors on CARLA-generated images of two different object classes and evaluate them on publicly available datasets. We provide access to our synthetic fire hydrant(3) and crosswalk4 datasets as well as provide step-by-step instructions(5) to generate custom datasets in CARLA.