Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
arXiv (Cornell University)(2023)
Abstract
By providing external information to large language models (LLMs), tool
augmentation (including retrieval augmentation) has emerged as a promising
solution for addressing the limitations of LLMs' static parametric memory.
However, how receptive are LLMs to such external evidence, especially when the
evidence conflicts with their parametric memory? We present the first
comprehensive and controlled investigation into the behavior of LLMs when
encountering knowledge conflicts. We propose a systematic framework to elicit
high-quality parametric memory from LLMs and construct the corresponding
counter-memory, which enables us to conduct a series of controlled experiments.
Our investigation reveals seemingly contradicting behaviors of LLMs. On the one
hand, different from prior wisdom, we find that LLMs can be highly receptive to
external evidence even when that conflicts with their parametric memory, given
that the external evidence is coherent and convincing. On the other hand, LLMs
also demonstrate a strong confirmation bias when the external evidence contains
some information that is consistent with their parametric memory, despite being
presented with conflicting evidence at the same time. These results pose
important implications that are worth careful consideration for the further
development and deployment of tool- and retrieval-augmented LLMs. Resources are
available at https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict.
MoreTranslated text
Key words
knowledge conflicts,language models,stubborn sloth
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined