Patch Inherent Feature Guided Mask Selection for Image Compression

Recent advances in Learned Image Compression (LIC) utilize the powerful Masked Image Modeling (MIM) framework (e.g., MAE). However, the random masking treats all patches equally, which could lead (i) missing critical information and (ii) poor robustness, since different patches may have different contributions to image reconstruction. The principle experiment shows that the patch-level inherent feature evaluation is highly correlated with the quality of the reconstructed image. Based on this key insight, we propose a simple mask selection approach based on the patch’s inherent feature. Specifically, we design plug-and-play frequency and spatial inherent feature modules to select masks, enhancing various MIMs based on the MAE model and achieving high-quality image reconstruction and strong robustness at high mask rates for data-intensive compression tasks.