Efficient Transfer Learning for Spectral Image Reconstruction from RGB Images
2020 IEEE Colombian Conference on Applications of Computational Intelligence (IEEE ColCACI 2020)(2020)
Abstract
Spectral image reconstruction from RGB images has emerged as a hot topic in the computer vision community due to easy-access and low-cost acquisition of the latter. The goal is to learn a non-linear mapping from 3-RGB bands to L spectral bands. With the growth of the available spectral datasets, this mapping has been learned using deep convolutional representations. However, these methods demand a large number of spectral images to train the net to obtain a good recovery. In contrast, the proposed process consists of a pre-training step where the weights of a convolutional neural network fit with a large amount of available RGB datasets without spectral mapping, taking into account the RGB system acquisition as a layer. Then, some layers of this pre-trained network are frozen to retrain it with the available spectral dataset to generate a spectral image with L bands. The proposed training scheme can be used with any pre-existing deep network that maps RGB to spectral images and it is here evaluated with a “U-net” architecture, and the RGB sensing is based on the Bayer filter pattern. The simulated and experimental data demonstrate the effectiveness of the proposed approach compared to training without transfer learning, showing a gain of up to 4 dB, with less spectral data.
MoreTranslated text
Key words
Spectral and RGB images,convolutional neural network,transfer learning
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined