Self-Alignment with Instruction Backtranslation
International Conference on Learning Representations(2023)
摘要
We present a scalable method to build a high quality instruction following
language model by automatically labelling human-written text with corresponding
instructions. Our approach, named instruction backtranslation, starts with a
language model finetuned on a small amount of seed data, and a given web
corpus. The seed model is used to construct training examples by generating
instruction prompts for web documents (self-augmentation), and then selecting
high quality examples from among these candidates (self-curation). This data is
then used to finetune a stronger model. Finetuning LLaMa on two iterations of
our approach yields a model that outperforms all other LLaMa-based models on
the Alpaca leaderboard not relying on distillation data, demonstrating highly
effective self-alignment.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要