CPNet: Continuity Preservation Network for infrared video colorization
Computer Vision and Image Understanding(2023)
Abstract
Infrared video colorization can significantly improve perceptual quality by predicting reasonable colors and restoring vivid details, especially in harsh environments. However, as an inconspicuous computer vision task, there is no specialized method. Besides, directly applying current grayscale colorization methods may generate structurally obscure and temporally inconsistent frames. In this paper, we design an infrared video colorization network CPNet aims to generate visually plausible and spatial–temporal consistent colorized videos. To achieve this, a feature fusion module and a hierarchical colorizer are designed to learn the importance of each consecutive frame and the local and global correlation of the integrated features, respectively. In addition, to consolidate the temporal consistency at a fine-grained level, we further introduce a composite loss function to narrow the distance between high-level feature representations while retaining pixel-wise correspondence. Moreover, a new metric named Mean Temporal Variation Similarity (MTVS) is proposed for effectively evaluating the degree of video continuity. Comprehensive experiments conducted on the KAIST dataset demonstrate the superiority of CPNet to produce more authentic colorized videos than state-of-the-art colorization methods. In terms of quantitative comparison, CPNet achieves improvements of at least 0.89 dB on PSNR, 0.016 on SSIM, and a significant promotion on MTVS. In addition, experiments conducted on the DAVIS dataset also prove the applicability of CPNet for grayscale video colorization task.
MoreTranslated text
Key words
41A05,41A10,65D05,65D17,Infrared video colorization,Convolutional neural network,Hybrid architecture
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined