Assessing Sample Quality via the Latent Space of Generative Models
arxiv(2024)
Abstract
Advances in generative models increase the need for sample quality
assessment. To do so, previous methods rely on a pre-trained feature extractor
to embed the generated samples and real samples into a common space for
comparison. However, different feature extractors might lead to inconsistent
assessment outcomes. Moreover, these methods are not applicable for domains
where a robust, universal feature extractor does not yet exist, such as medical
images or 3D assets. In this paper, we propose to directly examine the latent
space of the trained generative model to infer generated sample quality. This
is feasible because the quality a generated sample directly relates to the
amount of training data resembling it, and we can infer this information by
examining the density of the latent space. Accordingly, we use a latent density
score function to quantify sample quality. We show that the proposed score
correlates highly with the sample quality for various generative models
including VAEs, GANs and Latent Diffusion Models. Compared with previous
quality assessment methods, our method has the following advantages: 1)
pre-generation quality estimation with reduced computational cost, 2)
generalizability to various domains and modalities, and 3) applicability to
latent-based image editing and generation methods. Extensive experiments
demonstrate that our proposed methods can benefit downstream tasks such as
few-shot image classification and latent face image editing. Code is available
at https://github.com/cvlab-stonybrook/LS-sample-quality.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined