Decompositions of the mean continuous ranked probability score
arxiv(2023)
摘要
The continuous ranked probability score (crps) is the most commonly used
scoring rule in the evaluation of probabilistic forecasts for real-valued
outcomes. To assess and rank forecasting methods, researchers compute the mean
crps over given sets of forecast situations, based on the respective predictive
distributions and outcomes. We propose a new, isotonicity-based decomposition
of the mean crps into interpretable components that quantify miscalibration
(MSC), discrimination ability (DSC), and uncertainty (UNC), respectively. In a
detailed theoretical analysis, we compare the new approach to empirical
decompositions proposed earlier, generalize to population versions, analyse
their properties and relationships, and relate to a hierarchy of notions of
calibration. The isotonicity-based decomposition guarantees the nonnegativity
of the components and quantifies calibration in a sense that is stronger than
for other types of decompositions, subject to the nondegeneracy of empirical
decompositions. We illustrate the usage of the isotonicity-based decomposition
in case studies from weather prediction and machine learning.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要