Vision-Based Recognition of Construction Workers' Hand Signals

The development of robotic machines has shown the potential to promote automation and address issues in construction. One of the critical steps to make these machines work with onsite construction workers as teams is to provide a user-friendly interface to support their interactions. So far, little related work could be found in the construction domain. This paper tried to fill the gap and proposed a vision-based framework to recognize the construction workers' hand signals to interact with construction robotic machines. The framework includes three main components: (1) detection and tracking, (2) region cropping, and (3) hand signal recognition. Specifically, it started with the visual detection and tracking of a construction signalman from a video sequence. Based on the detection and tracking results, the regions of the signalman were cropped to form a hand signal recognition queue. Then, if a hand signal was detected, a pre-trained classifier would be used to identify the meaning of the signal. The framework was tested with two construction site videos. The test results indicated the effectiveness of the proposed framework on common construction hand signal recognition.