A Spatio-Temporal Interactive Network Based Sobel Guided Optical Flow for Facial Depression Analysis

The proficient utilization of machine learning has demonstrated the conspicuous presence of depressive features within facial region. Compared to dynamic descriptors, optical flow is more capable of capturing temporal nuance. However, the complexity of face makes it difficult to extract pure optical flow. In the paper, we propose a spatio-temporal interactive network (STI-Net) for facial depression analysis. By utilizing the SEInceptionV3 to construct the STI-Net backbone including temporal stream and spatial stream, it is more effective to capture facial spatio-temporal characteristics in parallel and achieve tight information interactivity. In the representation of temporal clues, the Sobel guided optical flow (SGOF) is introduced to enhance the edge, which can reduce facial complexity and simplify it to the contour variations. For the feature fusion, the adaptive weighting module (AWM) can assign different weights to temporal and spatial features to explore their correlations. Experiments on AVEC2014 datasets have shown that our method outperforms other visual-based methods in exploring and fusing facial features, and the proposed SGOF can provide guidance for the further application of optical flow in depression recognition.