Learning 6-DoF Fine-grained Grasp Detection Based on Part Affordance Grounding
CoRR(2023)
Abstract
Robotic grasping is a fundamental ability for a robot to interact with the
environment. Current methods focus on how to obtain a stable and reliable
grasping pose in object level, while little work has been studied on part
(shape)-wise grasping which is related to fine-grained grasping and robotic
affordance. Parts can be seen as atomic elements to compose an object, which
contains rich semantic knowledge and a strong correlation with affordance.
However, lacking a large part-wise 3D robotic dataset limits the development of
part representation learning and downstream applications. In this paper, we
propose a new large Language-guided SHape grAsPing datasEt (named LangSHAPE) to
promote 3D part-level affordance and grasping ability learning. From the
perspective of robotic cognition, we design a two-stage fine-grained robotic
grasping framework (named LangPartGPD), including a novel 3D part language
grounding model and a part-aware grasp pose detection model, in which explicit
language input from human or large language models (LLMs) could guide a robot
to generate part-level 6-DoF grasping pose with textual explanation. Our method
combines the advantages of human-robot collaboration and LLMs' planning ability
using explicit language as a symbolic intermediate. To evaluate the
effectiveness of our proposed method, we perform 3D part grounding and
fine-grained grasp detection experiments on both simulation and physical robot
settings, following language instructions across different degrees of textual
complexity. Results show our method achieves competitive performance in 3D
geometry fine-grained grounding, object affordance inference, and 3D part-aware
grasping tasks. Our dataset and code are available on our project website
https://sites.google.com/view/lang-shape
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined