Classifying Sport-Related Human Activity from Thermal Vision Sensors Using CNN and LSTM

In this work, we describe a classification of five sport-related human activities which are sensed by a thermal vision sensor. First, we have collected several sport sessions of an inhabitant while developing: push-ups, sit-ups, jumping jacks, squats and planks. Second, we develop an ad-hoc augmentation of data to increase the sturdiness of the data collection and reduce overfitting. Third, a Deep Learning model has been evaluated to compute a sequence of images from the user in order to estimate the activity. A CNN extracts relavant features from spatial domain and LSTM network models the sequence of images to compute the final classification. The results show an encouraging performance and quick learning capabilities.