GaitCTCG: cross-view gait recognition via cascaded residual temporal shift and comprehensive multi-granularity learning

Gait is one of the most popular biometrics for identity authentication today due to its noninvasive perception. Diverse spatial representations and temporal modeling are crucial information for gait recognition, especially under covariation conditions. However, most existing algorithms only focus on the specific temporal-scale modeling (i.e., short-term or long-term) and single-level or single-granularity (i.e., global or local) spatial representation; these algorithms lack flexibility and diversity for the extraction of features. To address this issue, we propose a cascaded residual temporal shift and comprehensive multi-granularity learning (GaitCTCG) network for gait recognition. Specifically, a cascaded residual temporal shift (CRTS) module was proposed to capture multiple receptive fields in the temporal dimension without any additional parameters or computational cost, thereby flexibly integrating features of different temporal scales. A comprehensive multi-granularity learning (CMGL) module was designed with a multi-layer multi-granularity scheme to extract and fuse comprehensive spatial representations at different scales, exploiting various visual details of the input. Furthermore, a micro gait energy generator (MGEG) was also developed to distill sequence representation, which refined the local temporal segments while preserving richer spatial information. Extensive experiments on two of the most popular public datasets demonstrated the state-of-the-art performance of our proposed method, which achieved rank-1 accuracies of 98.0%, 95.3%, and 84.4% in the normal walking (NM), bag carrying (BG), and coat-wearing (CL) scenarios on CASIA-B, and 91.2% on OUMVLP. The source code will be published at https://github.com/HUAFOR/GaitCTCG .