Exploiting Channel Similarity for Network Pruning
IEEE Transactions on Circuits and Systems for Video Technology(2023)
摘要
To address the limitations of existing pruning methods in practical applications, such as the necessity of training from scratch with sparsity regularization or complex data-driven optimization, we set out from a novel perspective to explore parameter redundancy and accelerate deep CNNs. Precisely, we argue that channels revealing similar feature information have functional overlap and that each such similarity group can be reduced to a few representatives with little impact on the representational power of the model. After deriving an effective metric for evaluating channel similarity via probabilistic modeling, we introduce a similarity-based pruning framework based on hierarchical clustering.In particular, the proposed algorithm can be directly applied to all kinds of pre-trained CNN models for better trade-offs between latency and accuracy. Moreover, rather than relying on a pre-defined target structure, it automatically discovers resource-efficient ones out of the original model under given budgets, which is in the same flavor as NAS. Extensive experiments on benchmark datasets well demonstrate the superior performance of our approach over prior arts. On ImageNet, our pruned ResNet-50 with 30% FLOPs reduced outperforms the original model. We further extend our algorithm to a GAN-based generative model and achieve 2x acceleration, showing its remarkable generalization capability and flexibility.
更多查看译文
关键词
Channel similarity,model acceleration,network pruning
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要