VGG16 and Bi-LSTM fused with an attention mechanism for human action recognition in infrared images

Action recognition has long been a popular subject of research in computer vision because of its wide prospects for application. Infrared videos are suitable for monitoring in any kind of weather and can ensure the privacy of the data. We propose a method of human action recognition in infrared videos by fusing the visual geometry group 16 (VGG16) and bi-directional long short-term memory (Bi-LSTM) with an attention mechanism. First, we extract infrared images from an infrared video and pre-process them. Second, we use the VGG16 model to extract the spatial features of the images through convolution and pooling, and we apply the Bi-LSTM fused with the attention mechanism to extract their temporal features. Finally, the two networks obtain the results of classification through the score fusion strategy at the decision level. The method is tested on various infrared datasets and the results show that it is effective.