Interframe Association of YOLO Bounding Boxes in the Presence of Camera Panning and Zooming

In this paper, we develop an approach for measurement-to-track association (M2TA) in the presence of (unknown) camera panning and zooming from drone-captured video. Standard M2TA methods assume that the target motion can be used to predict the “measurement association regions” for the bounding boxes. However, if there is a sudden state change due to camera shift (panning) and zooming, it will lead to incorrect associations and poor tracking results. To solve this, the zoom ratio and panning in 2D coordinates are used to describe the camera motion parameters in each frame. The estimated parameters are obtained by a grid search combined with global assignment or directly solved using the linear least squares method, which is also combined iteratively with assignment. The goal is to achieve correct M2TA by adjusting the predicted measurements using the estimated camera parameters. These “improved” predictions can also be used to update the target state with filtering algorithms. Frames with panning or/and zooming from real data are used to illustrate the effectiveness of the proposed methods and compared with the validation gate method based on inflated covariances.