Distillation-fusion-semantic unified driven network for infrared and visible image fusion

Infrared and visible image fusion is critical in computer vision applications to integrate complementary information from input images into a single enhanced representation. However, existing infrared and visible image fusion approaches frequently overlook the joint requirements of feature transmission and advanced semantics. Consequently, they suffer from low feature utilization and weak generalization ability. Additionally, these approaches tend to concentrate on capturing global or local features while not understanding features from the perspective of the frequency components. This limitation makes it challenging to capture local frequency information accurately. To tackle these challenges, a novel joint training framework called Distillation -FusionSegmentation (DFSFuse) addresses the challenges of feature transfer, integration, and semantic comprehension in image fusion. This framework integrates the capabilities of knowledge transfer and semantic reasoning. The teacher and student networks ensure feature acquisition and processing during distillation. Specifically, the teacher network comprehends features from both a global perspective and frequency components and transfers these features to the student network to facilitate and oversee the learning process. Student networks specialize in extracting and reconstructing features from various modalities while efficiently learning and utilizing intermediate and fused features. Segmentation networks are employed to identify semantic content, emphasizing regions with rich semantics. Furthermore, we introduce a distilled semantic loss function to facilitate feature transfer efficiently. The experimental results indicate that our approach yields promising outcomes, achieving plausible visual effects compared to existing methods.