SugarcaneGAN: A novel dataset generating approach for sugarcane leaf diseases based on lightweight hybrid CNN-Transformer network

Generative Adversarial Networks (GAN) were applied to provide methodological support for efficient sample expansion of crop disease features. Accurate extraction of leaf foreground scenes is crucial for generating highquality disease features. However, the reported GAN models, such as LeafGAN and STA-GAN, mainly use GradCAM to achieve leaf segmentation, which is only suitable for circular-like leaves under simple backgrounds, and perform unsatisfactory for striped sugarcane leaves under complex backgrounds. To address these problems, we have established a novel data augmentation model SugarcaneGAN with a proposed lightweight U-RSwinT as its leaf extraction module and generator. The proposed U-RSwinT combines the advantages of CNN and Swin Transformer. Two datasets of real sugarcane diseased leaves and healthy leaves were collected and several corresponding GAN-generated disease datasets were generated to train classification models in the downstream task. Experimental results show that U-RSwinT outperforms other modules, such as DeepLabV3, Swin-unet, etc., in leaf extraction accuracy as well as in lesion generation quality under various conditions. The mean FID score of the data generated by SugarcaneGAN was 24% and 34% lower than that of LeafGAN and CycleGAN, respectively, indicating much higher quality of the generated data of SugarcaneGAN. Moreover, SugarcaneGAN only required 51.1% of the training time of LeafGAN. Three classification models (ResNet50, SLViT, and ViT/B16) training from different GAN-generated datasets were further tested in the real sugarcane disease dataset, SugarcaneGAN brings significantly higher test accuracies for all three classification models. The ResNet50 model trained by the SugarcaneGAN-generated dataset has its test accuracy, precision, recall, specificity, and F1 score improved by 12.8%, 0.49%, 26.26%, 2.99%, and 0.1554, respectively, compared to that based on the second best LeafGANgenerated dataset. All the results show that SugarcaneGAN brings great improvement in the upstream task of data generation as well as in the downstream task of disease classification compared to the state-of-art GAN models, indicating great potential for leaf-diseased leaf image augmentation and in-situ leaf disease diagnosis.