Learning Visuotactile Skills with Two Multifingered Hands
CoRR(2024)
摘要
Aiming to replicate human-like dexterity, perceptual experiences, and motion
patterns, we explore learning from human demonstrations using a bimanual system
with multifingered hands and visuotactile data. Two significant challenges
exist: the lack of an affordable and accessible teleoperation system suitable
for a dual-arm setup with multifingered hands, and the scarcity of
multifingered hand hardware equipped with touch sensing. To tackle the first
challenge, we develop HATO, a low-cost hands-arms teleoperation system that
leverages off-the-shelf electronics, complemented with a software suite that
enables efficient data collection; the comprehensive software suite also
supports multimodal data processing, scalable policy learning, and smooth
policy deployment. To tackle the latter challenge, we introduce a novel
hardware adaptation by repurposing two prosthetic hands equipped with touch
sensors for research. Using visuotactile data collected from our system, we
learn skills to complete long-horizon, high-precision tasks which are difficult
to achieve without multifingered dexterity and touch feedback. Furthermore, we
empirically investigate the effects of dataset size, sensing modality, and
visual input preprocessing on policy learning. Our results mark a promising
step forward in bimanual multifingered manipulation from visuotactile data.
Videos, code, and datasets can be found at https://toruowo.github.io/hato/ .
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要