PEFormer: a pixel-level enhanced CNN-transformer hybrid network for face image super-resolution

In recent years, the rapid development of deep learning technologies has advanced the field of Face Super-Resolution (FSR), particularly with the introduction of numerous methods based on CNNs and Transformers. However, employing these architectures individually often overlooks the relationship between global and local information, resulting in a lack of detail and clarity in FSR images. To address this issue, we propose a novel pixel-enhanced CNN-Transformer hybrid network for FSR, called PEFormer. For the CNN branch, we design a DetailCNN that utilizes Central Cross Differential Convolution (CCDC) to extract gradient features from adjacent local regions of the face, and subsequent feature enhancement yields details-rich local features. For the Transformer branch, we design a Pixel-level Triple-path Transformer (PTT), which includes a global feature-extracting Transformer, a Pixel-Feature Attention Block (PFAB) that supplements high-frequency features and pixel information, and residual connections. These three combined paths may capture pixel-level long-range visual dependencies. The cascading of DetailCNN and PTT enables frequent interaction between local and global information, improving the extraction of deep features. Additionally, we employ a progressive image reconstruction structure according to different upscaling factors, reinforcing multi-scale feature information for high-quality reconstructed images. Moreover, considering the strong identity attributes of face images, an identity loss function is incorporated during the training to refine the details of facial components, bringing them closer to the target image and more aligned with human perception. Experiment results indicate that our proposed method effectively enhances facial feature details and improves FSR performance, showing improvements in both qualitative and quantitative metrics compared to existing methods.