FineDiffusion: Scaling up Diffusion Models for Fine-grained Image Generation with 10,000 Classes
CoRR(2024)
Abstract
The class-conditional image generation based on diffusion models is renowned
for generating high-quality and diverse images. However, most prior efforts
focus on generating images for general categories, e.g., 1000 classes in
ImageNet-1k. A more challenging task, large-scale fine-grained image
generation, remains the boundary to explore. In this work, we present a
parameter-efficient strategy, called FineDiffusion, to fine-tune large
pre-trained diffusion models scaling to large-scale fine-grained image
generation with 10,000 categories. FineDiffusion significantly accelerates
training and reduces storage overhead by only fine-tuning tiered class
embedder, bias terms, and normalization layers' parameters. To further improve
the image generation quality of fine-grained categories, we propose a novel
sampling method for fine-grained image generation, which utilizes
superclass-conditioned guidance, specifically tailored for fine-grained
categories, to replace the conventional classifier-free guidance sampling.
Compared to full fine-tuning, FineDiffusion achieves a remarkable 1.56x
training speed-up and requires storing merely 1.77
parameters, while achieving state-of-the-art FID of 9.776 on image generation
of 10,000 classes. Extensive qualitative and quantitative experiments
demonstrate the superiority of our method compared to other parameter-efficient
fine-tuning methods. The code and more generated results are available at our
project website: https://finediffusion.github.io/.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined