Expression-Latent-Space-guided GAN for Facial Expression Animation based on Discrete Labels

Facial expression animation aims to synthesize face images that correspond to the target expression in a continuum. This is a challenging task because the animation not only cares about the smooth transition in the generated sequence but also needs to take the facial expression and identity details into consideration. Most existing expression animation methods resort to continuous expression labels, e.g. Action Units (AUs) or landmark sequences. Compared with discrete expression labels, the annotations of a considerable part of them are ambiguous and prone to errors. However, how to animate facial expression conditioned on discrete expression labels is less investigated and existing methods cannot generate satisfactory facial details. To tackle these problems, we propose an end-to-end Expression-Latent-Space-guided Generative Adversarial Network (ELS-GAN) model, which utilizes discrete expression labels as input to generate images with expected expressions, and employs expression latent space learning to control the expression changing process. An expression ranking loss is also proposed to strengthen expression intensity learning during generation. Moreover, we put forward a Self-Attention Generator to synthesize face images with fine details by considering both local areas and the long-range dependency of different areas. Extensive experiments show that our method can generate continuous intermediate expression between source and target expressions only conditioned on discrete labels and superior results are achieved compared with state-of-the-art methods.