Towards an Integrated Autonomous Data-Driven Grasping System with a Mobile Manipulator

We present an integrated grasping system for a mobile manipulator to grasp an unknown object of interest (OI) in an unknown environment. The system autonomously scans its environment, models the OI, plans and executes a grasp, while taking into account base pose uncertainty. Due to inherent line of sight limitations in sensing, a single scan of the OI often does not reveal enough information to complete grasp analysis; as a result, our system autonomously builds a model of an object via multiple scans from different locations until a grasp can be performed. A volumetric next-best-view (NBV) algorithm is used to model an arbitrary object and terminates modeling when force closure for the gripper is satisfied. Two experiments are presented: i) modeling and registration error is reduced by selecting viewpoints with more scan overlap, and ii) model reconstruction and grasps are successfully achieved while experiencing base pose uncertainty.