Pose Transfer Using Multiple Input Images

One key challenge of pose transfer lies in its large variation and occlusion. Existing methods are basically aimed at the migration of the pose in a single input image. As a result, these methods are difficult to predict reasonable areas of invisibility and cannot decouple the shape and style of clothing. Although pose transfer method using single input image can generate the results with correct structure, it cannot keep the original details of the image. In order to tackle this challenge, we present a two-stage generative model designed for pose transfer using multiple input images. The initial stage involves Feature Extraction, succeeded by the Reposing Stage. During the Feature Extraction stage, pertinent features are extracted from each input image. Subsequently, in the Reposing Stage, we introduce a pose-conditioned transformer-based StyleGAN generator, adding residual module and fusion module at different levels of the generator. In this way, we can get the most relevant features from each input image for weighted fusion, thus improving the quality of the results. We show that our method compares favorably against those using single input image in both quantitative evaluation and visual comparison.