How faithful are RAG models? Quantifying the tug-of-war between RAG and LLMs' internal prior
arxiv(2024)
摘要
Retrieval augmented generation (RAG) is often used to fix hallucinations and
provide up-to-date knowledge for large language models (LLMs). However, in
cases when the LLM alone incorrectly answers a question, does providing the
correct retrieved content always fix the error? Conversely, in cases where the
retrieved content is incorrect, does the LLM know to ignore the wrong
information, or does it recapitulate the error? To answer these questions, we
systematically analyze the tug-of-war between a LLM's internal knowledge (i.e.
its prior) and the retrieved information in settings when they disagree. We
test GPT-4 and other LLMs on question-answering abilities across datasets with
and without reference documents. As expected, providing the correct retrieved
information fixes most model mistakes (94
reference document is perturbed with increasing levels of wrong values, the LLM
is more likely to recite the incorrect, modified information when its internal
prior is weaker but is more resistant when its prior is stronger. Similarly, we
also find that the more the modified information deviates from the model's
prior, the less likely the model is to prefer it. These results highlight an
underlying tension between a model's prior knowledge and the information
presented in reference documents.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要