Fully convolutional online tracking

Online learning has turned out to be effective for improving tracking performance. However, it could be simply applied for classification branch, but still remains challenging to adapt to regression branch due to its complex design and intrinsic requirement for high-quality online samples. To tackle this issue, we present the fully convolutional online tracking framework, coined as FCOT, and focus on enabling online learning for both classification and regression branches by using a target filter based tracking paradigm. Our key contribution is to introduce an online regression model generator (RMG) for initializing weights of the target filter in the regression branch with online samples, and then optimizing this target filter weights based on the ground-truth samples at the first frame. Specifically, we devise a simple fully online tracker, composed of a feature extraction backbone, an up-sampling decoder, a multi-scale classification branch, and an anchor-free regression branch. Thanks to the unique design of RMG, our FCOT can not only handle the target variation along temporal dimension, but also overcome the issue of error accumulation during the tracking procedure. In addition, due to its simplicity in design, our FCOT could be trained and deployed in a fully convolutional manner with a real-time running speed. Our FCOT achieves promising performance on seven benchmarks, including VOT2018, LaSOT, TrackingNet, GOT-10k, OTB100, UAV123, and NFS. Code and models of our FCOT are available at: https://github.com/MCG-NJU/FCOT.