Cross-Attention-Based Common and Unique Feature Extraction for Pansharpening

Pansharpening aims to integrate low-resolution multispectral (LRMS) images and panchromatic (PAN) images to obtain the high-resolution multispectral (HRMS) images. As PAN and LRMS images capture the same scene, they have some common information. Due to sensor characteristics’ differences, they also have unique information. Therefore, recent pansharpening methods separate the common and unique features to reduce redundancy. However, they are not explicitly equipped with specific modules to extract the common and unique features, which limits redundancy removal. To solve the problems, we propose a pansharpening method using a cross-attention-based module to specifically extract the common and unique feature maps from the MS and PAN images. In addition, mutual information (MI) maximization and minimization constraints are used to enforce the common and unique features, which can reduce the redundant information. Finally, these features are concatenated to generate the HRMS. Extensive experimental results on the QuickBird and Gaofen-2 datasets demonstrate that the proposed method outperforms other methods quantitatively and qualitatively.