Tiny Models are the Computational Saver for Large Models
CoRR(2024)
摘要
This paper introduces TinySaver, an early-exit-like dynamic model compression
approach which employs tiny models to substitute large models adaptively.
Distinct from traditional compression techniques, dynamic methods like
TinySaver can leverage the difficulty differences to allow certain inputs to
complete their inference processes early, thereby conserving computational
resources. Most existing early exit designs are implemented by attaching
additional network branches to the model's backbone. Our study, however,
reveals that completely independent tiny models can replace a substantial
portion of the larger models' job with minimal impact on performance. Employing
them as the first exit can remarkably enhance computational efficiency. By
searching and employing the most appropriate tiny model as the computational
saver for a given large model, the proposed approaches work as a novel and
generic method to model compression. This finding will help the research
community in exploring new compression methods to address the escalating
computational demands posed by rapidly evolving AI models. Our evaluation of
this approach in ImageNet-1k classification demonstrates its potential to
reduce the number of compute operations by up to 90
losses in performance, across various modern vision models. The code of this
work will be available.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要