Multi-granularity feature utilization network for cross-modality visible-infrared person re-identification

Cross-modality visible-infrared person re-identification (VI-ReID) aims to recognize images with the same identity between visible modality and infrared modality, which is a very challenging task because it not only includes the troubles of variations between cross-cameras in traditional person ReID, but also suffers from the huge differences between two modalities. Some existing VI-ReID methods extract the modality-shared information from global features through single-stream or double-stream networks, while ignoring the complementarity between fine-grained and coarse-grained information. To solve this problem, our paper designs a multi-granularity feature utilization network (MFUN) to make up for the lack of shared features between different modalities by promoting the complementarity between coarse-grained features and fine-grained features. Firstly, in order to learn fine-grained shared features better, we design a local feature constraint module, which uses both hard-mining triplet loss and heterogeneous center loss to constrain local features in the common subspace, so as to better promote intra-class compactness and inter-class differences at the level of sample and class center. Then, our method uses a multi-modality feature aggregation module for global features to fuse the information of two modalities to narrow the modality gap. Through the combination of these two modules, visible and infrared image features can be better fused, thus alleviating the problem of modality discrepancy and supplementing the lack of modality-shared information. Extensive experimental results on RegDB and SYSU-MM01 datasets fully prove that our proposed MFUN outperforms the state-of-the-art solutions. Our code is available at https://github.com/ZhangYinyinzzz/MFUN .