Text-based Talking Facial Synthesis for Virtual Host System
2022 International Conference on Culture-Oriented Science and Technology (CoST)(2022)
Abstract
With the prevailing of deep learning technology, automatic virtual image synthesis has made huge progress and the popularity of virtual portraits has been growing rapidly. Traditional virtual synthesis system rely on computer graphics method driven by motion capture of a real person, which need labor and equipment costs. In view of this, our paper proposes a virtual host synthesis method based on text driven to generate lip shape and facial animation(include eye movement and head pose) from a signal facial image of a virtual host. More precisely, we use three main modules to construct a virtual host synthesis system: a speech synthesis module based on Tacotron2, a speech to landmark points module to extract mixture landmarks movement to speech, and video generation module based on conditional generative adversarial network to generate video frames and realize time-continuous automatic sport news reporting.
MoreTranslated text
Key words
virtual portraits,virtual host,lip synthesis,talking face generation
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined