Uncertainty Characterization for 3D Object Detection Algorithms

Deep learning has long been the preferred method in acquiring accurate object detection in the context of autonomous driving. Research advancement in deep learning object detection keeps pushing the detection performance - typically measured by mean-average-precision (mAP) - to new records in open datasets like KITTI and NuScene. However, as the mAP measure of performance is based on a fixed threshold value of intersection of union (IOU), it overlooks the object localization error represented by the Euclidean distance between the detected object and ground truth. This paper presents a novel method to evaluate object detection algorithm performance by characterizing localization error and uncertainty. This study selects two different stereo vision algorithms and a corresponding Lidar algorithm for evaluation. The proposed method finds the uncertainty is more significant in the depth direction than in the lateral direction, giving the spatial uncertainty distribution an elliptical shape. The results also show that the error does not always increase monotonically with respect to the depth and is algorithm dependent. The uncertainty characteristics captured from this paper can improve object tracking and multi-sensor fusion performance.