Super-resolution to enhance low-resolution thermal facial expression images for thermal facial emotion recognition
semanticscholar(2021)
摘要
Facial emotion recognition from thermal images has gained more attention in recent years. Thermal cameras capture the heat emitted by objects and therefore thermal images are not sensitive to illumina tion changes. Furthermore, changes in temperature can indicate emotions and it is harder for humans to fake emotions in front of a thermal camera. However, a limitation is that, thermal cameras that cap ture highresolution images are expensive, and cheaper thermal cameras often capture images with a lowresolution and/or contaminated with noise and blur. Besides, lowresolution thermal images can also arise when images are captured from a far distance or from moving persons. When using these lowresolution thermal images for facial emotion recognition this can negatively influence the emotion classification accuracy. To tackle the problem of lowresolution thermal facial expression images, superresolution can be used. In this exploratory work, we propose the Thermal Face SuperResolution Network (TFSRNet) and the Thermal Face SuperResolution Generative Adversarial Network (TFSRGAN) to recover high resolution thermal facial expression images from lowresolution thermal facial expression images, with the goal to use the superresolved images for thermal facial emotion recognition. The architecture TF SRNet is optimized to minimize the mean squared error (MSE), which results in images with a high peak signaltonoise ratio (PSNR). However, these images often contain an unsatisfying perceptual quality. To generate highresolution images with a high perceptual quality we propose TFSRGAN. Both architectures use facial prior knowledge, such as facial landmark heatmaps and parsing maps, to enhance lowresolution thermal facial expression images. To emphasize the most important parts of each facial expression and to suppress irrelevant facial parts, we integrate the Convolutional Block Attention Module (CBAM) in both superresolution architectures. The proposed superresolution ar chitectures are used to enhance lowresolution thermal facial expression images, which are obtained with three different degradation models, namely bicubic downsampling (BI) on scale x2, x3 and x4, blurring followed by bicubic downsampling (BD) on scale x3 and bicubic downsampling followed by adding noise (DN) on scale x3. With an ablation study, the effectiveness of using facial prior knowledge and the attention mech anism CBAM for thermal superresolution is shown. When using facial prior knowledge and the at tention mechanism CBAM, the image quality of the superresolved images improves. Furthermore, experiments show that images enhanced by TFSRNet outperform bicubic interpolated images, for degradation models BI x4, BD x3 and DN x3. Using these superresolved images for thermal facial emotion recognition also leads to an increase of the emotion classification accuracy. In addition, im ages enhanced by TFSRGAN outperform bicubic interpolated images for degradation model DN x3. Although, this an exploratory work containing limitations, the experiments show the effectiveness of using facial prior knowledge and the attention mechanism CBAM for thermal facial expression super resolution. In addition, thermal face superresolution shows promising results for thermal facial emotion recognition where future work can build upon.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要