Neural Network Based Heterogeneous Sensor Fusion For Robot Motion Planning

This paper presents a neural network based heterogeneous sensor fusion approach towards real-time traversability estimation of mobile robots using sensor data. Even though significant advances have been made for autonomous navigation in structured terrain conditions, obtaining reliable traversability estimates for tracked vehicle navigation in challenging terrain conditions is still an open research problem. In this regard, we propose a neural network architecture capable of fusing depth images along with roll and pitch measurements on board the robot to perform traversability estimation. The proposed architecture is trained in a variety of simulated structured and unstructured environments. As such, the proposed architecture is capable of extracting the relevant features from the sensor measurements in a data driven manner as compared to existing heuristic based approaches that fail to generalize for different environmental conditions. The reliability of the traversability estimates provided by the trained architecture was validated in indoor and outdoor conditions using real sensor data. In addition, the feasibility of using the traversability estimates in incremental path planning was also demonstrated through simulation. For both applications the proposed approach provided compelling results. Inferences based on the results of the experiments along with directions for future research are also outlined.