CLLMs: Consistency Large Language Models
arxiv(2024)
摘要
Parallel decoding methods such as Jacobi decoding show promise for more
efficient LLM inference as it breaks the sequential nature of the LLM decoding
process and transforms it into parallelizable computation. However, in
practice, it achieves little speedup compared to traditional autoregressive
(AR) decoding, primarily because Jacobi decoding seldom accurately predicts
more than one token in a single fixed-point iteration step. To address this, we
develop a new approach aimed at realizing fast convergence from any state to
the fixed point on a Jacobi trajectory. This is accomplished by refining the
target LLM to consistently predict the fixed point given any state as input.
Extensive experiments demonstrate the effectiveness of our method, showing
2.4× to 3.4× improvements in generation speed while preserving
generation quality across both domain-specific and open-domain benchmarks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要