RGB-D Object Classification for Autonomous Driving Perception

Autonomous driving systems (ADS) comprise, essentially, sensory perception (including AI-ML-based techniques), localization, decision-making, and control. The cornerstone of an ADS is the sensory perception part, which is involved in most of the essential and necessary tasks for safe driving such as sensor-fusion, environment representation, scene understanding, semantic segmentation, object detection/recognition, and tracking. Multimodal sensor-fusion is an established strategy to enhance safety and robustness of perception systems in autonomous driving. In this work, a fusion of data from color-camera (RGB) and 3D-LIDAR (D-distance), henceforth designated RGB-D, will be particularly addressed, highlighting use-cases on road-users classification using deep learning. 3D-LIDAR data, in the form of point-cloud, can be processed directly by using the PointNet network or, alternatively, by using depth-maps, known as range-view representation, which is a suitable representation to train state-of-the-art Convolutional Neural Network (CNN) models and to make the combination with RGB-images more practical. Experiments are carried out using the KITTI dataset on object classification, i.e., vehicles, pedestrians, cyclists. We report extensive results in terms of classification performance of deep-learning models using RGB, 3D, and RGB-D representations. The results show that RGB-D models have better performance in comparison with 3D and range-view models but, in some circumstances RGB-only achieved superior performance.