A multimodal 2D Convolutional Neural Network for Multiple Sclerosis Lesion Detection

In this study, an automated machine learning approach for the segmentation of MS lesions from multi- modal magnetic resonance images (mmMRI) is presented. The method is based on a U-Net like convolutional neural network (CNN) for 2D slice-based segmentation of 3D brain MRI volumes. The different modalities are encoded in sep- arate downsampling channels. The skip connections input feature maps to multi-scale feature fusion blocks at every stage of the network. These are followed by multi-scale feature upsampling blocks, which use the information from lesion shape and location. The CNN is evaluated on two publicly available datasets: The ISBI 2015 longitudinal MS lesion segmentation challenge dataset which contains 14 MS patients and the MICCAI 2016 MSSEG challenge dataset consisting of 15 MS patients. Regarding the ISBI Challenge, the proposed method was among the top performing ap- proaches to which open-access papers are available. The MICCAI dataset served to evaluate the robustness of the architecture against scanner variability and to show the improvements in performance after transfer learning.