Efficient Continual Learning with Low Memory Footprint For Edge Device
arxiv(2024)
摘要
Continual learning(CL) is a useful technique to acquire dynamic knowledge
continually. Although powerful cloud platforms can fully exert the ability of
CL,e.g., customized recommendation systems, similar personalized requirements
for edge devices are almost disregarded. This phenomenon stems from the huge
resource overhead involved in training neural networks and overcoming the
forgetting problem of CL. This paper focuses on these scenarios and proposes a
compact algorithm called LightCL. Different from other CL methods bringing huge
resource consumption to acquire generalizability among all tasks for delaying
forgetting, LightCL compress the resource consumption of already generalized
components in neural networks and uses a few extra resources to improve memory
in other parts. We first propose two new metrics of learning plasticity and
memory stability to seek generalizability during CL. Based on the discovery
that lower and middle layers have more generalizability and deeper layers are
opposite, we Maintain Generalizability by freezing the lower and
middle layers. Then, we Memorize Feature Patterns to stabilize the
feature extracting patterns of previous tasks to improve generalizability in
deeper layers. In the experimental comparison, LightCL outperforms other SOTA
methods in delaying forgetting and reduces at most 6.16×
memory footprint, proving the excellent performance of LightCL in efficiency.
We also evaluate the efficiency of our method on an edge device, the Jetson
Nano, which further proves our method's practical effectiveness.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要