Gesture Evolution-Tracking Nonverbal Communication of Creative Design Teams

Communication of creative ideas is ambiguous and are often complimented with nonverbal forms of communication such as gestures or sketches. In-person team communication can take advantage of creating shared gestures and sketches more easily than in the growing remote format of hybrid work places. To understand differences in communication, we sought to track and analyze gestural communication of in-person creative design teams (CDT) in order to compare against remote CDT. Past work used Mechanical Turk to analyze video data, to our knowledge, we are the first attempt in automating the hand coding approach to analyzing video data. Using data collected on CDT, we trained a convolutional neural network to identify and flag deictic (pointing) gestures (DG). We trained our model and tested our model against two different time points separated by one month. This model could ideally be used on nascent CDTs to help them improve in their communication and overall team performance and for companies to understand how their in-person and remote CDTs work.