Towards Mixed Mode Biomarkers: Combining Structural and Functional Information by Deep Learning

The transference of information between entities is one of the most popular application of deep learning. It has been used to generate a stylized version of an image by combining a source image to another that determines the style of the final result. In the field of neuroimaging, different modalities are frequently available, providing structural or functional information. Those modalities are usually analyzed separately, although it is possible to jointly use features extracted from structural and functional neuroimage to improve the classification performance in Computer Aided Diagnosis (CAD) tools. In this paper we propose a method based on the principles of neural style transfer to combine information from Magnetic resonance Imaging (MRI) and Positron Emission Tomography (PET), generating a new image that contains structural and functional information. The usefulness of this method has been assessed with images from the Alzheimer Disease Neuroimaging Initiative, demonstrating that using the new mixed mode image outperforms the classification accuracy obtained by individual MRI or PET images.