Deep learning-based facial emotion recognition exploiting self-supervised pre-training and feature pyramid

Abstract Facial emotion recognition (FER), a technology designed to automatically identify the emotional state of an individual based on the features of his face, is a thriving area in human-computer interaction and affective computing. Among all the FER techniques, the deep learning model, especially the convolutional neural network is more successful. In this paper, various networks are tested firstly with different optimizers based on the VGG baseline. After that, according to the results of the test, it seems that VGG19 is the best choice for the backbone network. Following that, a feature pyramid network added to the backbone model is considered to improve the model, which, however, ends up with even worse accuracy. To find another way of improvement, the attention is thrown to the unsupervised contrastive model, which is built based on the momentum contrast with VGG19 being the backbone network, culminating in an increase in the accuracy from 71.94% to 72.11% in the end.