AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising
arxiv(2024)
摘要
Diffusion models have garnered significant interest from the community for
their great generative ability across various applications. However, their
typical multi-step sequential-denoising nature gives rise to high cumulative
latency, thereby precluding the possibilities of parallel computation. To
address this, we introduce AsyncDiff, a universal and plug-and-play
acceleration scheme that enables model parallelism across multiple devices. Our
approach divides the cumbersome noise prediction model into multiple
components, assigning each to a different device. To break the dependency chain
between these components, it transforms the conventional sequential denoising
into an asynchronous process by exploiting the high similarity between hidden
states in consecutive diffusion steps. Consequently, each component is
facilitated to compute in parallel on separate devices. The proposed strategy
significantly reduces inference latency while minimally impacting the
generative quality. Specifically, for the Stable Diffusion v2.1, AsyncDiff
achieves a 2.7x speedup with negligible degradation and a 4.0x speedup with
only a slight reduction of 0.38 in CLIP Score, on four NVIDIA A5000 GPUs. Our
experiments also demonstrate that AsyncDiff can be readily applied to video
diffusion models with encouraging performances. The code is available at
https://github.com/czg1225/AsyncDiff.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要