A joint optimization scheme to combine different levels of features for face recognition with makeup changes

We address the problem of makeup face recognition. Our main idea is to incorporate different levels of features into a joint optimization framework. Specifically, we combine both mid-level (e.g. attributes) and low-level features to obtain a new representation for a better matching between makeup and non-makeup faces. Previous studies have discovered the influence of cosmetics on face recognition, and some methods have been developed to address makeup face recognition. However, it is for the first time to integrate different levels of features to address this challenging problem. To minimize the differences between makeup and non-makeup faces, transformations are learned in a joint optimization framework which combines different levels of features. Experimental results demonstrate that our proposed approach outperforms the state-of-the-art methods.