Artistic image synthesis from unsupervised segmentation maps

We present a framework for artwork image synthesis from unsupervised segmentation maps input and style images. The output has style consistency with style images and the semantic structure from the corresponding segmentation label. Existing methods of transferring semantic labels to painting images require large amounts of manual segmentation pairs for training. To address the issue, we use unsupervised segmentation maps to build on training pairs and learn the generator with the proposed spatially adaptive instance normalization block. Our method exploits the style consistency and semantic consistency loss functions to reduce the artifact in synthetic images. Extensive experiments in several image translation tasks show the effectiveness of our method in generating an image with both structures of segmentation and style of exemplar image.