Joint Facial Action Unit Recognition and Self-Supervised Optical Flow Estimation
Pattern Recognition Letters(2024)
摘要
Facial action unit (AU) recognition and optical flow estimation are two highly correlated tasks, since optical flow can provide motion information of facial muscles to facilitate AU recognition. However, most existing AU recognition methods handle the two tasks independently by offline extracting optical flow as auxiliary information or directly ignoring the use of optical flow. In this paper, we propose a novel end -to -end joint framework of AU recognition and optical flow estimation, in which the two tasks contribute to each other. Moreover, due to the lack of optical flow annotations in AU datasets, we propose to estimate optical flow in a self -supervised manner. To regularize the self -supervised estimation of optical flow, we propose an identical mapping constraint for the optical flow guided image warping process, in which the estimated optical flow between two same images is required to not change the image during warping. Experiments demonstrate that our framework (i) outperforms most of the state-of-the-art AU recognition methods on the challenging BP4D and GFT benchmarks, and (ii) also achieves competitive self -supervised optical flow estimation performance.
更多查看译文
关键词
Joint framework,Facial AU recognition,Self-supervised optical flow estimation,Identical mapping constraint
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要