Self-Evaluation of Large Language Model based on Glass-box Features
arxiv(2024)
摘要
The proliferation of open-source Large Language Models (LLMs) underscores the
pressing need for evaluation methods. Existing works primarily rely on external
evaluators, focusing on training and prompting strategies. However, a crucial
aspect - model-aware glass-box features - is overlooked. In this study, we
explore the utility of glass-box features under the scenario of
self-evaluation, namely applying an LLM to evaluate its own output. We
investigate various glass-box feature groups and discovered that the softmax
distribution serves as a reliable indicator for quality evaluation.
Furthermore, we propose two strategies to enhance the evaluation by
incorporating features derived from references. Experimental results on public
benchmarks validate the feasibility of self-evaluation of LLMs using glass-box
features.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要