Multi-Sensor Fusion for Vehicle-to-Vehicle Cooperative Localization with Object Detection and Point Cloud Matching

Accurate vehicle pose is fundamental information required by automated driving systems. However, complicated driving environments and sensor failures have constrained onboard sensor-based single-vehicle localization precision. With the development of cooperative driving automation, the information from surrounding vehicles in the Vehicle-to-Everything (V2V) network offers remarkable potential to boost the ego vehicle’s localization performance. In this paper, we propose a cooperative vehicle localization framework based on multi-sensor fusion that uses shared information from multi-agents, leveraging point cloud feature matching and object detection. The ego vehicle’s detection system can determine the relative pose between the ego vehicle and the corresponding surrounding vehicles based on data from the LiDAR sensor. However, the accuracy of the pose information derived directly from deep-learning-based object detection is limited. Thus, a relative pose refining method is proposed to further improve the relative pose by applying a point cloud matching technique based on a normal distribution transformation approach. Meanwhile, to reduce the data transmission load, we extract only the edge and plane features from the surrounding vehicle’s LiDAR scan and exclude the remaining point cloud. Additionally, the shared information is fused into the ego vehicle’s INS-based localization system, which enables continuous and high-frequency localization output within a Kalman filter framework. To make the fusion algorithm more adaptive to different relative pose noise levels, a measurement quality evaluation rule is designed. Real-world vehicular experiments show that the proposed algorithm can improve localization accuracy by at least 35% compared to the traditional range-based cooperative localization method.