Scaling Multimodal Planning: Using Experience and Informing Discrete Search

Robotic manipulation is inherently continuous, but typically has an underlying discrete structure, such as if an object is grasped. Many problems like these are multimodal , such as pick-and-place tasks where every object grasp and placement is a mode . Multimodal problems require finding a sequence of transitions between modes—for example, a particular sequence of object picks and placements. However, many multimodal planners fail to scale when motion planning is difficult (e.g., in clutter) or the task has a long horizon (e.g., rearrangement). This work presents solutions for multimodal scalability in both these areas. For motion planning, we present an experience-based planning framework alef which reuses experience from similar modes both online and from training data. For task satisfaction, we present a layered planning approach that uses a discrete lead to bias search toward useful mode transitions, informed by weights over mode transitions. Together, these contributions enable multimodal planners to tackle complex manipulation tasks that were previously infeasible or inefficient, and provide significant improvements in scenes with high-dimensional robots.