MidFusNet: Mid-dense Fusion Network for Multi-modal Brain MRI Segmentation

The fusion of multi-modality information has proved effective at improving the segmentation results of targeted regions (e.g., tumours, lesions or organs) of medical images. In particular, layer-level fusion represented by DenseNet has demonstrated a promising level of performance for various medical segmentation tasks. Using stroke and infant brain segmentation as example of ongoing challenging applications involving multi-modal images, we investigate whether it is possible to create a more effective of parsimonious fusion architecture based on the state-of-art fusion network - HyperDenseNet. Our hypothesis is that by fully fusing features throughout the entire network from different modalities, this not only increases network computation complexity but also interferes with the unique feature learning of each modality. Nine new network variants involving different fusion points and mechanisms are proposed. Their performances are evaluated on public datasets including iSeg-2017 and ISLES15-SSIS and an acute stroke lesion dataset collected by medical professionals. The experiment results show that of the nine proposed variants, the 'mid-dense' fusion network (named as MidFusNet) is able to achieve a performance comparable to the state-of-art fusion architecture, but with a much more parsimonious network (i.e., 3.5 million parameters less compared to the baseline network for three modalities).