Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models
arXiv (Cornell University)(2023)
Abstract
The generative process of Diffusion Models (DMs) has recently set
state-of-the-art on many AI generation benchmarks. Though the generative
process is traditionally understood as an "iterative denoiser", there is no
universally accepted language to describe it. We introduce a novel perspective
to describe DMs using the mathematical language of memory retrieval from the
field of energy-based Associative Memories (AMs), making efforts to keep our
presentation approachable to newcomers to both of these fields. Unifying these
two fields provides insight that DMs can be seen as a particular kind of AM
where Lyapunov stability guarantees are bypassed by intelligently engineering
the dynamics (i.e., the noise and step size schedules) of the denoising
process. Finally, we present a growing body of evidence that records DMs
exhibiting empirical behavior we would expect from AMs, and conclude by
discussing research opportunities that are revealed by understanding DMs as a
form of energy-based memory.
MoreTranslated text
Key words
uncanny resemblances,diffusion models,memory
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined