CT Synthesis from Multi-Sequence MRI Using Adaptive Fusion Network

Objective: To investigate a method using multi-sequence magnetic resonance imaging (MRI) to synthesize computed tomography (CT) for MRI-only radiation therapy.Approach: We proposed an adaptive multi-sequence fusion network (AMSF-Net) to exploit both voxel-and context-wise cross-sequence correlations from multiple MRI sequences to synthesize CT using element-and patch-wise fusions, respectively. The element-and patch-wise fusion feature spaces were combined, and the most representative features were selected for modeling. Finally, a densely connected convolutional decoder was applied to utilize the selected features to produce synthetic CT images.Main results: This study includes a total number of 90 patients' T1-weighted MRI, T2-weighted MRI and CT data. The AMSF-Net reduced the average mean absolute error (MAE) from 52.88-57.23 to 49.15 HU, increased the peak signal-to-noise ratio (PSNR) from 24.82-25.32 to 25.63 dB, increased the structural similarity index mea-sure (SSIM) from 0.857-0.869 to 0.878, and increased the dice coefficient of bone from 0.886-0.896 to 0.903 compared to the other three existing multi-sequence learning models. The improvements were statistically sig-nificant according to two-tailed paired t-test. In addition, AMSF-Net reduced the intensity difference with real CT in five organs at risk, four types of normal tissue and tumor compared with the baseline models. The MAE de-creases in parotid and spinal cord were over 8% and 16% with reference to the mean intensity value of the corresponding organ, respectively. Further, the qualitative evaluations confirmed that AMSF-Net exhibited su-perior structural image quality of synthesized bone and small organs such as the eye lens.Significance: The proposed method can improve the intensity and structural image quality of synthetic CT and has potential for use in clinical applications.