Low Light Image Enhancement by Multispectral Fusion and Convolutional Neural Networks

In this paper, we propose low light image enhancement by multispectral fusion and convolutional neural networks (CNNs). We adopt multispectral fusion of color (RGB) and near infrared (NIR) images for low light image enhancement based on pyramid feature selection and attention map. The proposed fusion network consists of two subnetworks: denoising and fusion. Since low light RGB images contain severe noise with detail loss, we first utilize a denoising subnetwork to preprocess RGB images. In the denoising subnetwork, we use concat operation to prevent the loss of image features during training. We independently train the denoising subnetwork because denoising datasets are easy to obtain. After denoising, we use gamma correction to enhance the denoised low light image. Finally, we perform the fusion subnetwork for hidden texture recovery based on pyramid feature selection and attention map. To build the fusion subnetwork, we synthesize a low light image dataset based on smoothing and gamma correction. We generate the ground truth for training by adding the details of the NIR images into the smoothed RGB images. Experimental results show that the proposed fusion method generates high quality images with little noise, fine details and good colors.