End to End Framework for CNN Acceleration on FPGAs with Dynamic Algorithm Mapping.

As CNNs are becoming more diverse with respect to per-layer computation characteristics, per-layer strategy selection and fine-grained tuning are required to achieve low end-to-end latency. DYNAMAP is a framework for efficiently selecting algorithms for different layers and re-using a unified accelerator. However, the end-to-end deployment of DYNAMAP faces some challenges in terms of productivity and portability. In this work, we develop an API for automatically extracting the ONNX graph from high-level programming libraries such as Pytorch and Tensorflow. We further integrate this with the front-end of DYNAMAP to increase its portability. We then define an HLS template of the accelerator supporting three parallel algorithm choices for convolution operations to increase the productivity of generating hardware. Using three state-of-the-art CNNs, we demonstrate that our framework is able to automatically deploy CNN models to achieve minimum latency which incurs very low overhead in the deployment process.