Can Large Language Models Capture Public Opinion about Global Warming? An Empirical Assessment of Algorithmic Fidelity and Bias
CoRR(2023)
摘要
Large language models (LLMs) have demonstrated their potential in social
science research by emulating human perceptions and behaviors, a concept
referred to as algorithmic fidelity. This study assesses the algorithmic
fidelity and bias of LLMs by utilizing two nationally representative climate
change surveys. The LLMs were conditioned on demographics and/or psychological
covariates to simulate survey responses. The findings indicate that LLMs can
effectively capture presidential voting behaviors but encounter challenges in
accurately representing global warming perspectives when relevant covariates
are not included. GPT-4 exhibits improved performance when conditioned on both
demographics and covariates. However, disparities emerge in LLM estimations of
the views of certain groups, with LLMs tending to underestimate worry about
global warming among Black Americans. While highlighting the potential of LLMs
to aid social science research, these results underscore the importance of
meticulous conditioning, model selection, survey question format, and bias
assessment when employing LLMs for survey simulation. Further investigation
into prompt engineering and algorithm auditing is essential to harness the
power of LLMs while addressing their inherent limitations.
更多查看译文
关键词
large language models,language models,global warming
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要