MSSTNet: A Multi-Scale Spatio-Temporal CNN-Transformer Network for Dynamic Facial Expression Recognition
arxiv(2024)
摘要
Unlike typical video action recognition, Dynamic Facial Expression
Recognition (DFER) does not involve distinct moving targets but relies on
localized changes in facial muscles. Addressing this distinctive attribute, we
propose a Multi-Scale Spatio-temporal CNN-Transformer network (MSSTNet). Our
approach takes spatial features of different scales extracted by CNN and feeds
them into a Multi-scale Embedding Layer (MELayer). The MELayer extracts
multi-scale spatial information and encodes these features before sending them
into a Temporal Transformer (T-Former). The T-Former simultaneously extracts
temporal information while continually integrating multi-scale spatial
information. This process culminates in the generation of multi-scale
spatio-temporal features that are utilized for the final classification. Our
method achieves state-of-the-art results on two in-the-wild datasets.
Furthermore, a series of ablation experiments and visualizations provide
further validation of our approach's proficiency in leveraging spatio-temporal
information within DFER.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要