Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving

Simulators play a crucial role in autonomous driving, offering significant time, cost, and labor savings. Over the past few years, the number of simulators for autonomous driving has grown substantially. However, there is a growing concern about the validity of algorithms developed and evaluated in simulators, indicating a need for a thorough analysis of the development status of the simulators. To address existing gaps in research, this paper undertakes a comprehensive review of the history of simulators, proposes a utility-based taxonomy, and investigates the critical issues within open-source simulators. Analysis of the past thirty years' development trajectory reveals a trend characterized by an increase in open-source simulators and an expansion of their functionality scope. The categorization of simulators based on feature functionalities delineates five primary classes: traffic flow, sensory data, driving policy, vehicle dynamics, and comprehensive simulators. Furthermore, the paper identifies critical unresolved issues in open-source simulators, including concerns regarding the fidelity of sensory data, representation of traffic scenarios, and accuracy in vehicle dynamics simulation, all of which have the potential to undermine experimental confidence. Additionally, challenges in data format inconsistency, labor-intensive map construction processes, sluggish step updating, and insufficient support for Hardware-In-the-Loop testing are discussed as hindrances to experimental efficiency. In light of these findings, the survey furnishes task-oriented recommendations to aid in the selection of simulators, taking into account factors such as accessibility, maintenance status, and quality, while highlighting the inherent limitations of existing open-source simulators in validating algorithms and facilitating real-world experimentation.