Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations
CoRR(2024)
Abstract
A major barrier towards the practical deployment of large language models
(LLMs) is their lack of reliability. Three situations where this is
particularly apparent are correctness, hallucinations when given unanswerable
questions, and safety. In all three cases, models should ideally abstain from
responding, much like humans, whose ability to understand uncertainty makes us
refrain from answering questions we don't know. Inspired by analogous
approaches in classification, this study explores the feasibility and efficacy
of abstaining while uncertain in the context of LLMs within the domain of
question-answering. We investigate two kinds of uncertainties, statistical
uncertainty metrics and a distinct verbalized measure, termed as In-Dialogue
Uncertainty (InDU). Using these uncertainty measures combined with models with
and without Reinforcement Learning with Human Feedback (RLHF), we show that in
all three situations, abstention based on the right kind of uncertainty measure
can boost the reliability of LLMs. By sacrificing only a few highly uncertain
samples we can improve correctness by 2
correctly identifying unanswerable questions and increase safety by 70
99
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined