The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation
CoRR(2023)
摘要
Large Language Models (LLMs) encapsulate vast amounts of knowledge but still
remain vulnerable to external misinformation. Existing research mainly studied
this susceptibility behavior in a single-turn setting. However, belief can
change during a multi-turn conversation, especially a persuasive one.
Therefore, in this study, we delve into LLMs' susceptibility to persuasive
conversations, particularly on factual questions that they can answer
correctly. We first curate the Farm (i.e., Fact to Misinform) dataset, which
contains factual questions paired with systematically generated persuasive
misinformation. Then, we develop a testing framework to track LLMs' belief
changes in a persuasive dialogue. Through extensive experiments, we find that
LLMs' correct beliefs on factual knowledge can be easily manipulated by various
persuasive strategies.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要