GIoU-CLOCs: Import Generalized Intersection Over Union-Involved Object-Detection Tasks Based on Lidar and Camera

In recent years, the application of LIDAR has become much more extensive, especially in object detection. While laser researches have encouraging performance in detection, they are typically based on a single modality, being unable to collect information from other modalities. In this paper, we introduce a late fusion way to fuse data from LIDAR and RGB camera. For the disorder of laser, we introduce polynomial functions into the 3D network, which enable the network to take higher-order moments of a given shape into account. Considering the geometric and semantic consistency, we fuse point clouds and images to generate more accurate 3D and 2D detection results. Finally, we address the weaknesses of the intersection over union in the fusion network, employing a generalized version as both a new loss and a new metric. The experimental evaluation of the challenging KITTI object detection benchmark shows significant improvements, especially in the birds’ eye view, which shows the feasibility and applicability of our work.