Out of Sight, Still in Mind: Reasoning and Planning about Unobserved Objects with Video Tracking Enabled Memory Models
arxiv(2023)
摘要
Robots need to have a memory of previously observed, but currently occluded
objects to work reliably in realistic environments. We investigate the problem
of encoding object-oriented memory into a multi-object manipulation reasoning
and planning framework. We propose DOOM and LOOM, which leverage transformer
relational dynamics to encode the history of trajectories given partial-view
point clouds and an object discovery and tracking engine. Our approaches can
perform multiple challenging tasks including reasoning with occluded objects,
novel objects appearance, and object reappearance. Throughout our extensive
simulation and real-world experiments, we find that our approaches perform well
in terms of different numbers of objects and different numbers of distractor
actions. Furthermore, we show our approaches outperform an implicit memory
baseline.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要