Exploration of Balanced Design in Resource-Constrained Edge Device for Efficient CNNs

Convolutional Neural Networks (CNNs) have been widely used in various image recognition applications due to their high precision and high versatility. In order to improve real-time performance, many efficient CNNs with low computational complexity have been proposed. However, compared with the reduction in the amount of calculation, the demand for memory access has not decreased simultaneously. How to balance the utilization of various resources has become a challenge. This brief deeply deconstructs the CNN network and analyzes the relationship among computational resources, on-chip memory, off-chip bandwidth, and throughput. With a given bandwidth, we propose a cross-layer scheduling to minimize the on-chip memory occupancy, which can still fully utilize the computational resources. We implement a flexible FPGA-based hardware accelerator that can deploy the proposed design. According to the experiments on the MobileNetV2, our design reduces the on-chip memory usage by 91.7% and achieves state-of-the-art performance.