Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Tasks
arxiv(2024)
摘要
Trustworthy prediction in Deep Neural Networks (DNNs), including Pre-trained
Language Models (PLMs) is important for safety-critical applications in the
real world. However, DNNs often suffer from uncertainty estimation, such as
miscalibration. In particular, approaches that require multiple stochastic
inference can mitigate this problem, but the expensive cost of inference makes
them impractical. In this study, we propose k-Nearest Neighbor Uncertainty
Estimation (kNN-UE), which is an uncertainty estimation method that uses the
distances from the neighbors and label-existence ratio of neighbors.
Experiments on sentiment analysis, natural language inference, and named entity
recognition show that our proposed method outperforms the baselines or recent
density-based methods in confidence calibration, selective prediction, and
out-of-distribution detection. Moreover, our analyses indicate that introducing
dimension reduction or approximate nearest neighbor search inspired by recent
kNN-LM studies reduces the inference overhead without significantly degrading
estimation performance when combined them appropriately.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要