Performance Modeling of Computer Vision-based CNN on Edge GPUs.

Convolutional Neural Networks (CNNs) are widely used in various fields nowadays, particularly for computer vision applications. Edge platforms have drawn tremendous attention from academia and industry due to their ability to improve execution time and preserve privacy. However, edge platforms struggle to satisfy CNN’s needs due to their computation and energy constraints. It is then challenging to find the most efficient CNN that respects accuracy, time, energy, and memory footprint constraints for a target edge platform. Furthermore, given the size of the design space of CNNs and hardware platforms, performance evaluation of CNNs entails several challenges and efforts. Consequently, the designers need tools to quickly explore the large design space and select the CNN that offers the best performance trade-off for a set of hardware (HW) platforms. This paper proposes a Machine Learning (ML) based modeling approach for CNN performances on edge GPU-based platforms for vision applications. We implement and compare five (5) of the most successful ML algorithms for accurate and rapid CNN performances predictions on three (3) different edge GPUs in image classification. Experimental results demonstrate the robustness and usefulness of our proposed methodology. For three of the five ML algorithms, namely XGBoost, Random Forest, and Ridge Polynomial regression, an average error of 11%, 6%, and 8%, have been obtained for CNN inference execution time, power consumption, and memory usage, respectively.