Experimental Design for Active Transductive Inference in Large Language Models
CoRR(2024)
摘要
One emergent ability of large language models (LLMs) is that query-specific
examples can be included in the prompt at inference time. In this work, we use
active learning for adaptive prompt design and call it Active In-context Prompt
Design (AIPD). We design the LLM prompt by adaptively choosing few-shot
examples from a training set to optimize performance on a test set. The
training examples are initially unlabeled and we obtain the label of the most
informative ones, which maximally reduces uncertainty in the LLM prediction. We
propose two algorithms, GO and SAL, which differ in how the few-shot examples
are chosen. We analyze these algorithms in linear models: first GO and then use
its equivalence with SAL. We experiment with many different tasks in small,
medium-sized, and large language models; and show that GO and SAL outperform
other methods for choosing few-shot examples in the LLM prompt at inference
time.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要