Diagnosis of brain diseases in fusion of neuroimaging modalities using deep learning: A review

•Review of papers on the diagnosis of brain diseases using the fusion of neuroimaging modalities is presented.•For each brain disease, the most appropriate neuroimaging modalities and best combinations are introduced.•Advantages and disadvantages of fusion levels and different modalities are introduced.•Details of various types of deep neural networks used for the task at hand are also examined.•Finally, future directions and challenges in the field are discussed.