Never-Ending Embodied Robot Learning
arxiv(2024)
摘要
Relying on large language models (LLMs), embodied robots could perform
complex multimodal robot manipulation tasks from visual observations with
powerful generalization ability. However, most visual behavior-cloning agents
suffer from manipulation performance degradation and skill knowledge forgetting
when adapting into a series of challenging unseen tasks. We here investigate
the above challenge with NBCagent in embodied robots, a pioneering
language-conditioned Never-ending Behavior-Cloning agent, which can continually
learn observation knowledge of novel robot manipulation skills from
skill-specific and skill-shared attributes. Specifically, we establish a
skill-specific evolving planner to perform knowledge decoupling, which can
continually embed novel skill-specific knowledge in our NBCagent agent from
latent and low-rank space. Meanwhile, we propose a skill-shared semantics
rendering module and a skill-shared representation distillation module to
effectively transfer anti-forgetting skill-shared knowledge, further tackling
catastrophic forgetting on old skills from semantics and representation
aspects. Finally, we design a continual embodied robot manipulation benchmark,
and several expensive experiments demonstrate the significant performance of
our method. Visual results, code, and dataset are provided at:
https://neragent.github.io.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要