AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy
CoRR(2024)
摘要
Large language models (LLMs) show impressive capabilities, matching and
sometimes exceeding human performance in many domains. This study explores the
potential of LLMs to augment judgement in forecasting tasks. We evaluated the
impact on forecasting accuracy of two GPT-4-Turbo assistants: one designed to
provide high-quality advice ('superforecasting'), and the other designed to be
overconfident and base-rate-neglecting. Participants (N = 991) had the option
to consult their assigned LLM assistant throughout the study, in contrast to a
control group that used a less advanced model (DaVinci-003) without direct
forecasting support. Our preregistered analyses reveal that LLM augmentation
significantly enhances forecasting accuracy by 23
assistants, compared to the control group. This improvement occurs despite the
superforecasting assistant's higher accuracy in predictions, indicating the
augmentation's benefit is not solely due to model prediction accuracy.
Exploratory analyses showed a pronounced effect in one forecasting item,
without which we find that the superforecasting assistant increased accuracy by
43
augmentation disproportionately benefits less skilled forecasters, degrades the
wisdom-of-the-crowd by reducing prediction diversity, or varies in
effectiveness with question difficulty. Our findings do not consistently
support these hypotheses. Our results suggest that access to an LLM assistant,
even a biased one, can be a helpful decision aid in cognitively demanding tasks
where the answer is not known at the time of interaction.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要