Integrating digital twin and computer vision system for efficient pick-and-place operation using Tecnomatix Process Simulate

The rapid growth in the automation industry calls for reducing the time required to do repetitive tasks while minimizing the error caused by the human involved in the process. Pick-and-place is one of the most performed tasks by a cobot, but humans are required to manually provide the coordinates of the object in the program. In order to successfully pick an object from a new location, it is necessary for a human operator to not only input the new location into the program but also verify the feasibility of the operation. This includes checking for potential issues such as collisions or insufficient reach of the cobot. In addition, the operator must assess the environmental conditions and ensure that the robot's end effector is suitable for the object's size, shape, and weight. These factors are crucial in determining the success of the pick and place operation. This study introduces a digital twin concept with a computer vision system to solve these issues. Computer vision system detects the object and generates the coordinate every time the object appears in a location, whether the location is the same or new. The object detection task is performed using MATLAB. Then, a digital twin of the actual cobot is utilized to simulate the generated pick-and-place process and check whether the process is feasible. The digital twin of the KUKA LBR iiwa 7-R800 cobot was generated using Tecnomatix Process Simulate, and it was used to perform the pick-and-place operation. Integration between software was achieved and automation using python worked flawlessly. Automation made process noticeably easier to work in dynamic environment.