Learning the external and internal priors for multispectral and hyperspectral image fusion

Recently, multispectral image (MSI) and hyperspectral image (HSI) fusion has been a popular topic in high-resolution HSI acquisition. This fusion leads to a challenging underdetermined problem, which image priors are used to regularize, aiming at improving fusion accuracy. To fully exploit HSI priors, this paper proposes two kinds of priors, i.e., external priors and internal priors, to regularize the fusion problem. An external prior represents the general image characteristics and is learned from abundant training data by using a Gaussian denoising convolutional neural network (CNN) trained in the additional gray images. An internal prior represents the unique characteristics of the HSI and MSI to be fused. To learn the external prior, we first segment the MSI into several superpixels and then enforce a low-rank constraint for each superpixel, which can well model local similarities in the HSI. In addition, to model a low-rank property in the spectral mode, the high-resolution HSI is decomposed into a low-rank spectral basis and abundances. Finally, we formulate the fusion as an external and internal prior-regularized optimization problem, which is efficiently tackled through the alternating direction method of multipliers. Experiments on simulated and real datasets demonstrate the superiority of the proposed method.