The Hidden Linear Structure in Score-Based Models and its Application.
CoRR(2023)
摘要
Score-based models have achieved remarkable results in the generative
modeling of many domains. By learning the gradient of smoothed data
distribution, they can iteratively generate samples from complex distribution
e.g. natural images.
However, is there any universal structure in the gradient field that will
eventually be learned by any neural network? Here, we aim to find such
structures through a normative analysis of the score function.
First, we derived the closed-form solution to the scored-based model with a
Gaussian score. We claimed that for well-trained diffusion models, the learned
score at a high noise scale is well approximated by the linear score of
Gaussian. We demonstrated this through empirical validation of pre-trained
images diffusion model and theoretical analysis of the score function. This
finding enabled us to precisely predict the initial diffusion trajectory using
the analytical solution and to accelerate image sampling by 15-30\% by skipping
the initial phase without sacrificing image quality. Our finding of the linear
structure in the score-based model has implications for better model design and
data pre-processing.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要