Off-Line Programming Framework for Sorting Task Based on Human-Demonstration

Sorting tasks are typical applications in the product and industrial domains. When facing new settings, such as different types of objects and their positions, the system has to be reprogrammed by experienced engineers, which decreases production efficiency and increases labour and downtime costs. In this paper, an off-line programming framework for sorting tasks based on human demonstration has been developed, taking advantage of machine vision and programming-by-demonstration concepts. Only one eye-to-hand depth camera has been used to record both the demonstrated trajectory and the corresponding relationship of the sorting task. A feature generalization algorithm has also been proposed to ensure that the system is able to reproduce the sorting task using both the key features of the trajectory and the sorting features. Finally, tracking, object and target matching, and generalization experiments have been conducted using the JAKA Zu7 robot with different types of fruit models and target boxes. The results show that the proposed framework is able to record the sorting trajectory at a frequency of 10 Hz, acquire and maintain the sorting relationship after new demonstration, and automatically reproduce the sorting task in new settings.