Controllable fundus image generation based on conditional generative adversarial networks with mask guidance

Fundus image generation can serve training data diversity as well as clinical and medical education. To solve the problem of the controllability and flexibility of fundus image generation with multiple lesion features, we propose a controllable fundus image generation model (CFIGGAN) based on conditional generative adversarial networks (GAN) for medical data augmentation. The least square loss and the perceptual loss term are added to the final loss to make the generated images more realistic, and the spectral normalization is used as the normalization method of the discriminator to make the training process more stable. In the two-stage training of the model, the vascular tree image concatenates the real and generated images as positive and negative samples to train the model. CFIGGAN can generate diseased fundus images by using the annotations of vascular tree, field of vision(FOV), DR-related lesions as input and controlling the morphology of four types of lesions. Qualitative experimental evaluation shows that the fundus images generated by our model are clear and realistic and close to the real image data distribution. Quantitative experimental evaluation shows that the combination of the spectral norm and the perceptual loss can improve visual observation and quantitative indices, and data augmentation by image generation can further increase the classification accuracy. More importantly, CFIGGAN achieve the controllability of fundus image generation corresponding to DR-related lesions, and the proposed method can be extended to medical images generation of other diseases for broader prospects.