BitsFusion: 1.99 bits Weight Quantization of Diffusion Model
arxiv(2024)
Abstract
Diffusion-based image generation models have achieved great success in recent
years by showing the capability of synthesizing high-quality content. However,
these models contain a huge number of parameters, resulting in a significantly
large model size. Saving and transferring them is a major bottleneck for
various applications, especially those running on resource-constrained devices.
In this work, we develop a novel weight quantization method that quantizes the
UNet from Stable Diffusion v1.5 to 1.99 bits, achieving a model with 7.9X
smaller size while exhibiting even better generation quality than the original
one. Our approach includes several novel techniques, such as assigning optimal
bits to each layer, initializing the quantized model for better performance,
and improving the training strategy to dramatically reduce quantization error.
Furthermore, we extensively evaluate our quantized model across various
benchmark datasets and through human evaluation to demonstrate its superior
generation quality.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined