Learning to Simulate Labelled Datasets with an Image-Level Content Consistent Graph Constraint

Recent years have witnessed the explosive growth of data annotation due to the deep learning algorithm's thirst for labelled datasets. To ease the workload of data annotation, researchers begin to concentrate on automatic dataset generation, i.e. dataset synthesis via virtual engine like Unity. However, the large domain gap between the target and simulated images hinders the great potential of these methods. In this paper, we propose a novel method to simulate labelled datasets under an image-level content consistent graph constraint. A detection model is applied to extract the semantic content of target and simulated images, and the corresponding scene graphs are constructed to compare the differences between them. This method not only utilizes the high-dimensional features of the images to reduce the content gap between target and simulated datasets, but also reserves the differentiability of the whole learning pipeline. As a result, we are able to automatically simualte labelled datasts in an end-to-end manner, and the models trained on the simulated datasets behave the best against existing methods.