Trimodal Fusion Network Combined Global-Local Feature Extraction Strategy and Spatial-Frequency Fusion Strategy.

Two or more images are fused into one to complement each other’s information, which is conducive to assisting doctors in locating disease types, conditions, and lesions. Most of the existing methods focus on the fusion of two modalities, and in fact, it is very common for the disease that requires the fusion of trimodal to assist diagnosis. This paper proposes a high-precision trimodal medical image fusion network. According to the information characteristics of anatomical images and functional images in medical images, we designed the global texture module (GTM) and the local detail module (LDM) for feature extraction simultaneously. The fusion strategy fully combined the advantages of spatial domain and frequency domain to retain more complete texture detail information and global contour information. At the same time, the multi-attention mechanism is adopted to extract more effective depth features and more accurate location information. Experimental results show that the proposed method is effective in both subjective vision and objective index evaluation.