3D object detection and state estimation method based on stereo vision and LIDAR fusion

Environmental perception is the foundation of autonomous driving, and a single sensor cannot handle the complex urban road environment efficiently. In this paper, we propose a vehicle object shape, position and motion state estimation method based on stereo vision and LiDAR fusion. First, images and point clouds of LiDAR are detected using YOLOv4 and Point-GNN, respectively. Secondly, to estimate the position and shape of the object, the paper proposes an estimation method using 3D points of image object and object type. Finally, the image and LiDAR objects are converted to the same vehicle coordinate system using the calibrated external parameters, the IOU and Global Nearest Neighbor(GNN) data association method is used to match the image object and the LiDAR object, and the matched object are fused to obtain the accuracy shape-position of the object, and the object is tracked using the extended Kalman filter (EKF) algorithm. This paper is validated on the KITTI dataset and real road scenes. The experimental results show that the average errors of object shape, position and velocity in 20m are 4.33%,4.41% and 2.34%, respectively.