Simulation-Based Calibration Checking for Bayesian Computation: The Choice of Test Quantities Shapes Sensitivity
arxiv(2022)
摘要
Simulation-based calibration checking (SBC) is a practical method to validate
computationally-derived posterior distributions or their approximations. In
this paper, we introduce a new variant of SBC to alleviate several known
problems. Our variant allows the user to in principle detect any possible issue
with the posterior, while previously reported implementations could never
detect large classes of problems including when the posterior is equal to the
prior. This is made possible by including additional data-dependent test
quantities when running SBC. We argue and demonstrate that the joint likelihood
of the data is an especially useful test quantity. Some other types of test
quantities and their theoretical and practical benefits are also investigated.
We provide theoretical analysis of SBC, thereby providing a more complete
understanding of the underlying statistical mechanisms. We also bring attention
to a relatively common mistake in the literature and clarify the difference
between SBC and checks based on the data-averaged posterior. We support our
recommendations with numerical case studies on a multivariate normal example
and a case study in implementing an ordered simplex data type for use with
Hamiltonian Monte Carlo. The SBC variant introduced in this paper is
implemented in the 𝚂𝙱𝙲 R package.
更多查看译文
关键词
calibration checking,bayesian computation,simulation-based
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要