Teaching Language Models to Self-Improve through Interactive Demonstrations
arxiv(2023)
Abstract
The self-improving ability of large language models (LLMs), enabled by
prompting them to analyze and revise their own outputs, has garnered
significant interest in recent research. However, this ability has been shown
to be absent and difficult to learn for smaller models, thus widening the
performance gap between state-of-the-art LLMs and more cost-effective and
faster ones. To reduce this gap, we introduce TriPosT, a training algorithm
that endows smaller models with such self-improvement ability, and show that
our approach can improve a LLaMA-7b's performance on math and reasoning tasks
by up to 7.13
model to interact with LLMs to collect feedback and improvements on its own
generations. We then replay this experience to train the small model. Our
experiments on four math and reasoning datasets show that the interactive
experience of learning from and correcting its own mistakes is crucial for
small models to improve their performance.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined