Efficient FPGA design for Convolutions in CNN based on FFT-pruning

Fast algorithms of convolution, such as Winograd and fast Fourier transformation (FFT), have been widely used in many FPGA-based CNN accelerators to reducing the complexity of multiplication. The core idea for those fast algorithms is reducing the number of multiplication at the cost of more additions. However, increased additions take up a significant portion in the whole LUT resources in many cases, which forms a new bottleneck in the corresponding hardware design. In this paper, we theoretically analyze the relationship between the reduced multiplications and the increased additions, and propose an reduced complexity fast FFT convolution algorithm by intelligently employing the FFT-pruning method to remove redundant additions. Compared with the state-of-the-art algorithm, our algorithm can reduce more than 50% of additions. Moreover, the proposed algorithm has better numerical accuracy and comparable multiplication complexity compared to the most efficient Winograd algorithm. Additionally, an efficient reconfigurable architecture of the proposed algorithm is also developed to accelerate convolutional layers with various kernel sizes. Implemented with Xilinx ZC706, the proposed architecture achieves 200.6 GOPS on convolutional layers of ResNet-50 with 61% higher resources efficiency with respect to LUT consumption compared to prior arts.