Are Large Language Models (LLMs) Good Social Predictors?
CoRR(2024)
摘要
The prediction has served as a crucial scientific method in modern social
studies. With the recent advancement of Large Language Models (LLMs), efforts
have been made to leverage LLMs to predict the human features in social life,
such as presidential voting. These works suggest that LLMs are capable of
generating human-like responses. However, we find that the promising
performance achieved by previous studies is because of the existence of input
shortcut features to the response. In fact, by removing these shortcuts, the
performance is reduced dramatically. To further revisit the ability of LLMs, we
introduce a novel social prediction task, Soc-PRF Prediction, which utilizes
general features as input and simulates real-world social study settings. With
the comprehensive investigations on various LLMs, we reveal that LLMs cannot
work as expected on social prediction when given general input features without
shortcuts. We further investigate possible reasons for this phenomenon that
suggest potential ways to enhance LLMs for social prediction.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要