Deep Learning Based Method For Pruning Deep Neural Networks

In order to implement Deep Neural Networks (DNNs) into mobile devices, network pruning has been widely explored for lightening the complexity of Deep Neural Networks in terms of computational cost and parameter-storage load. In this paper, we propose a novel filter-level pruning method which utilizes a deep learning method to pursue compact DNNs. Specifically, we use a DNN model to extract features from the filters at first. Then, we employ a clustering algorithm to force the extracted features roll into different groups. By mapping the clustering results to the filters, we get the "similarity" relationships among the filters. At last, we keep the filter which is closest to the centroid in each cluster, prune out the others, and retrain the pruned DNN model. Compared with previous methods that employ heuristic ways on filters directly or selecting shallow features from filters manually, our method takes advantages of the deep learning method which can represent the raw filters in a more precise way. Experimental results show that our method outperforms several state-of-the-art pruning methods with negligible accuracy loss.