Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging
arxiv(2023)
摘要
Selective experience replay is a popular strategy for integrating lifelong
learning with deep reinforcement learning. Selective experience replay aims to
recount selected experiences from previous tasks to avoid catastrophic
forgetting. Furthermore, selective experience replay based techniques are model
agnostic and allow experiences to be shared across different models. However,
storing experiences from all previous tasks make lifelong learning using
selective experience replay computationally very expensive and impractical as
the number of tasks increase. To that end, we propose a reward
distribution-preserving coreset compression technique for compressing
experience replay buffers stored for selective experience replay.
We evaluated the coreset compression technique on the brain tumor
segmentation (BRATS) dataset for the task of ventricle localization and on the
whole-body MRI for localization of left knee cap, left kidney, right
trochanter, left lung, and spleen. The coreset lifelong learning models trained
on a sequence of 10 different brain MR imaging environments demonstrated
excellent performance localizing the ventricle with a mean pixel error distance
of 12.93 for the compression ratio of 10x. In comparison, the conventional
lifelong learning model localized the ventricle with a mean pixel distance of
10.87. Similarly, the coreset lifelong learning models trained on whole-body
MRI demonstrated no significant difference (p=0.28) between the 10x compressed
coreset lifelong learning models and conventional lifelong learning models for
all the landmarks. The mean pixel distance for the 10x compressed models across
all the landmarks was 25.30, compared to 19.24 for the conventional lifelong
learning models. Our results demonstrate that the potential of the
coreset-based ERB compression method for compressing experiences without a
significant drop in performance.
更多查看译文
关键词
selective experience replay compression,reinforcement learning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要