Interpretable Robotic Manipulation from Language
CoRR(2024)
Abstract
Humans naturally employ linguistic instructions to convey knowledge, a
process that proves significantly more complex for machines, especially within
the context of multitask robotic manipulation environments. Natural language,
moreover, serves as the primary medium through which humans acquire new
knowledge, presenting a potentially intuitive bridge for translating concepts
understandable by humans into formats that can be learned by machines. In
pursuit of facilitating this integration, we introduce an explainable behavior
cloning agent, named Ex-PERACT, specifically designed for manipulation tasks.
This agent is distinguished by its hierarchical structure, which incorporates
natural language to enhance the learning process. At the top level, the model
is tasked with learning a discrete skill code, while at the bottom level, the
policy network translates the problem into a voxelized grid and maps the
discretized actions to voxel grids. We evaluate our method across eight
challenging manipulation tasks utilizing the RLBench benchmark, demonstrating
that Ex-PERACT not only achieves competitive policy performance but also
effectively bridges the gap between human instructions and machine execution in
complex environments.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined