Controlled Text Generation via Language Model Arithmetic
ICLR 2024(2023)
Abstract
As Large Language Models (LLMs) are deployed more widely, customization with
respect to vocabulary, style and character becomes more important. In this work
we introduce model arithmetic, a novel inference framework for composing and
biasing LLMs without the need for model (re)training or highly specific
datasets. In addition, the framework allows for more precise control of
generated text than direct prompting and prior controlled text generation (CTG)
techniques. Using model arithmetic, we can express prior CTG techniques as
simple formulas and naturally extend them to new and more effective
formulations. Further, we show that speculative sampling, a technique for
efficient LLM sampling, extends to our setting. This enables highly efficient
text generation with multiple composed models with only marginal overhead over
a single model. Our empirical evaluation demonstrates that model arithmetic
allows fine-grained control of generated text while outperforming
state-of-the-art on the task of toxicity reduction.
MoreTranslated text
Key words
Controlled text generation,LLM,Natural Language Processing
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined