Optimization-based Motion Planning with Human in The Loop for Non-Prehensile Manipulation

We are interested in the reaching through clutter problem where a number of robots are trying to reach for a goal objects from the back of cluttered shelves. We investigate the performance increase that can be achieved by using a human-inthe-loop to guide these robots. The Reaching Through Clutter problems are difficult for fully autonomous planners as they have to search for a solution in a high-dimensional space. Furthermore, physics simulators suffer from the uncertainty problem where a valid trajectory in simulation can be invalid when executing the trajectory in the real-world. We propose an online-replanning method with human-in-the-loop to tackle these problems. This system enables a robot to plan and execute a trajectory autonomously, but also to seek high-level input from a human operator if needed. This method aims to minimize the human effort required, thereby increasing the number of robots that can be guided in parallel by a single human operator.