Multi-band Convolutional Neural Network for Efficient Utilization of Model Parameters

A convolutional layer of a traditional convolutional neural network (CNN) does not ensure the extraction of complementary bands of the input data. Thus a significant amount of redundancy is observed among different convolutional filters. Here, we propose a novel architecture design framework called ‘Multi-band CNN’ to efficiently utilize model parameters in a CNN. The framework generates four filters from a single filter by varying their frequency responses, extracting four complementary bands of the input data without increasing the parameter count. This leads to higher parameter utilization and results in a compact network with reduction in trainable parameter count but with close to the same accuracy as the base model. We perform experiments using residual networks (ResNet-32, ResNet-56, and ResNet-110) on datasets like CIFAR-10 and CIFAR-100. Our results show improved classification accuracy for CIFAR-10 when introducing a multi-band layer in the first convolutional layer, while there is no significant drop in accuracy for CIFAR-100. The performance is better when replacing the first convolutional layer instead of the last one, indicating that low-level features generated by sub-band filtering are more beneficial to the network than high-level features provided by filter banks at the final layer. The proposed Multi-band CNN framework offers a potential solution for reducing the number of filters required to train and the computational complexity of generating feature maps in CNNs, while maintaining or even improving classification accuracy.