Hybrid attention mechanism of feature fusion for medical image segmentation

Traditional convolution neural networks (CNN) have achieved good performance in multi-organ segmentation of medical images. Due to the lack of ability to model long-range dependencies and correlations between image pixels, CNN usually ignores the information of channel dimension. To further improve the performance of multi-organ segmentation, a hybrid attention mechanism model is proposed. First, a CNN was used to extract multi-scale feature maps and fed into the Channel Attention Enhancement Module (CAEM) to selectively pay attention to target organs in medical images, and the Transformer encoded tokenized image patches from CNN feature maps as the input sequence to model long-range dependencies. Second, the decoder upsampled the output from Transformer and fused with the CAEM features in multi-scale through skip connections. Finally, we introduced a Refinement Module (RM) after the decoder to improve feature correlations of the same organ and the feature discriminability between different organs. The model outperformed on dice coefficient (%) and hd95 on both the synapse multi-organ segmentation and cardiac diagnosis challenge datasets. The hybrid attention mechanisms exhibited high efficiency and high segmentation accuracy in medical images. We design a hybrid attention mechanism networks to segment multi-organ in medical image. On the one hand, self-attention and channel attention are integrated to capture global and local features which adapt to the different contrast among organs; on the other hand, a refinement module refine is constructed to refine organ details to further optimize segmentation accuracy. Our model helps to identify organs and boundaries in medical images, significantly improving the segmentation accuracy of medical images.image