Could It Be Generated? Towards Practical Analysis of Memorization in Text-To-Image Diffusion Models
CoRR(2024)
Abstract
The past few years have witnessed substantial advancement in text-guided
image generation powered by diffusion models. However, it was shown that
text-to-image diffusion models are vulnerable to training image memorization,
raising concerns on copyright infringement and privacy invasion. In this work,
we perform practical analysis of memorization in text-to-image diffusion
models. Targeting a set of images to protect, we conduct quantitive analysis on
them without need to collect any prompts. Specifically, we first formally
define the memorization of image and identify three necessary conditions of
memorization, respectively similarity, existence and probability. We then
reveal the correlation between the model's prediction error and image
replication. Based on the correlation, we propose to utilize inversion
techniques to verify the safety of target images against memorization and
measure the extent to which they are memorized. Model developers can utilize
our analysis method to discover memorized images or reliably claim safety
against memorization. Extensive experiments on the Stable Diffusion, a popular
open-source text-to-image diffusion model, demonstrate the effectiveness of our
analysis method.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined