Energy-Efficient Deep Neural Network Optimization via Pooling-Based Input Masking

Deep Neural Networks (DNNs) are increasingly deployed in battery-powered and resource-constrained devices. However, the most accurate DNNs usually require millions of parameters and operations, making them computation-heavy and energy-expensive, so it is an important topic to develop energy efficient DNN models. In this paper, we present an efficient DNN training framework under energy constraint to improve the energy efficiency of DNN inference. The key idea of this research is inspired by the observation that the input data of DNNs is usually inherently sparse and such sparsity can be exploited by sparse tensor DNN accelerators to eliminate ineffectual data access and compute. Therefore, we can enhance the inference accuracy within the energy budget by strategically controlling the sparsity of the input data. We build an energy consumption model for the sparse tensor DNN accelerator to quantify the inference energy consumption from the perspective of data access and data processing. In particular, we define a metric (named sporadic degree) to characterise the influence of the number of sporadic values in the sparse input on the energy consumption of data access for the sparse tensor DNN accelerator. Based on the proposed quantitative energy consumption model, we present an efficient pooling-based input mask training algorithm to optimize the energy efficiency of DNN inference by enhancing the input sparsity and reducing the number of sporadic values in the masked input. Experiments show that compared with the state-of-the-art methods, our proposed method can achieve higher inference accuracy with lower energy consumption and storage requirement owing to higher sparsity and lower sporadic degree of the masked input.