Deep calibration for broadband multispectral filter array

Broadband multispectral filter array (BMSFA) has emerged as an attractive alternative for spectral imaging due to its compactness and high light throughput. It compresses the multispectral data cube to 2D measurements, and then we reconstruct the data cube from the measurements with its pre-calibrated spectral response. In practice, the BMSFA spectral response is usually calibrated using ultra-narrow filters along wavelength with high cost. In addition, the process introduces noise and inter-spectral crosstalk that would severely degrade reconstruction quality. In this work, we report a novel calibration technique using deep learning called deep calibration. The technique generates different spectral illumination and collects sets of measurements of BMSFA camera and corresponding true spectra. In this way, a more accurate characterization of BMSFA spatial-spectral modulation can be obtained. Furthermore, a reconstruction network following hybrid CNN-Vit architecture was employed to learn the demodulation process from the collected dataset. Then, using this network as a decoder, the scene's hyperspectral data can be accurately reconstructed from the measurements. Extensive experiments validated that the reported technique performs with high efficiency and accuracy in both calibration and reconstruction of BMSFA.