Feature-Based Knowledge Distillation for Infrared Small Target Detection

Infrared small target detection is an extremely challenging task because of its low resolution, noise interference, and the weak thermal signal of small targets. Nevertheless, despite these difficulties, there is a growing interest in this field due to its significant application value in areas such as military, security, and unmanned aerial vehicles. In light of this, we propose a feature-based knowledge distillation method (IRKD) for infrared small target detection, which can efficiently transfer detailed knowledge to students. The key idea behind IRKD is to assign varying importance to the features of teachers and students in different areas during the distillation process. Treating all features equally would negatively impact the distillation results. Therefore, we have developed a Unified Channel-Spatial Attention (UCSA) module that adaptively enhances the crucial learning areas within the features. The experimental results show that compared to other knowledge distillation methods, our student detector achieved significant improvement in mean average precision (mAP). For example, IRKD improves ResNet50-based RetinaNet from 50.5% to 59.1% mAP and improves ResNet50-based FCOS from 53.7% to 56.6% on Roboflow.