Knowing what to know: Implications of the choice of prior distribution on the behavior of adaptive design optimization
arxiv(2023)
摘要
Adaptive design optimization (ADO) is a state-of-the-art technique for
experimental design (Cavagnaro, Myung, Pitt, Kujala, 2010). ADO dynamically
identifies stimuli that, in expectation, yield the most information about a
hypothetical construct of interest (e.g., parameters of a cognitive model). To
calculate this expectation, ADO leverages the modeler's existing knowledge,
specified in the form of a prior distribution. Informative priors align with
the distribution of the focal construct in the participant population. This
alignment is assumed by ADO's internal assessment of expected information gain.
If the prior is instead misinformative, i.e., does not align with the
participant population, ADO's estimates of expected information gain could be
inaccurate. In many cases, the true distribution that characterizes the
participant population is unknown, and experimenters rely on heuristics in
their choice of prior and without an understanding of how this choice affects
ADO's behavior.
Our work introduces a mathematical framework that facilitates investigation
of the consequences of the choice of prior distribution on the efficiency of
experiments designed using ADO. Through theoretical and empirical results, we
show that, in the context of prior misinformation, measures of expected
information gain are distinct from the correctness of the corresponding
inference. Through a series of simulation experiments, we show that, in the
case of parameter estimation, ADO nevertheless outperforms other design
methods. Conversely, in the case of model selection, misinformative priors can
lead inference to favor the wrong model, and rather than mitigating this
pitfall, ADO exacerbates it.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要