A comprehensive study on codebook-based feature fusion for gait recognition

Vision-based person identification using gait is one of the important and challenging tasks in the fields of computer vision and machine learning. It has received significant research efforts in the past two decades due to its several benefits. It is non-invasive and can be performed at a distance without active collaboration from users. The identification can be performed from low-resolution videos using simple instrumentation. The conventional gait recognition approaches usually operate on the sequence of extracted human silhouettes. They derive several gait-related features from the segmented binary energy maps of the walker. However, such processes are sensitive to variations in the silhouette shapes, thus limiting their efficacy. Codebook-based feature encoding techniques have been proven to be effective and reported state-of-the-art recognition results on several visual datasets such as action recognition, image, and video classification, etc. The whole process usually follows the pipeline of pattern recognition which mainly consists of five steps: (i) local feature extraction, (ii) feature pre-processing, (iii) codebook computation, (iv) feature encoding, and (v) classification. Each step in the pipeline plays a crucial role in recognition accuracy. Since the visual gait sequences comprise different walking patterns of the subjects due to variations in their static appearance and motion dynamics, several features are extracted to encode this information. Finally, they are fused to recognize the identity of the subject. This paper presents a comprehensive study of codebook-based approaches, explains all the steps in the encoding of visual gait sequences, and uncovers some good practices to obtain state-of-the-art recognition results. In particular, we investigated two different local features to encode the static appearance and motion information of the walker, and twelve kinds of feature encoding methods. An extensive evaluation of these encoding methods is carried out on a large benchmark CASIA-B gait database and their performance comparison is presented.