Vanilla Bayesian Optimization Performs Great in High Dimensions
arxiv(2024)
摘要
High-dimensional problems have long been considered the Achilles' heel of
Bayesian optimization algorithms. Spurred by the curse of dimensionality, a
large collection of algorithms aim to make it more performant in this setting,
commonly by imposing various simplifying assumptions on the objective. In this
paper, we identify the degeneracies that make vanilla Bayesian optimization
poorly suited to high-dimensional tasks, and further show how existing
algorithms address these degeneracies through the lens of lowering the model
complexity. Moreover, we propose an enhancement to the prior assumptions that
are typical to vanilla Bayesian optimization algorithms, which reduces the
complexity to manageable levels without imposing structural restrictions on the
objective. Our modification - a simple scaling of the Gaussian process
lengthscale prior with the dimensionality - reveals that standard Bayesian
optimization works drastically better than previously thought in high
dimensions, clearly outperforming existing state-of-the-art algorithms on
multiple commonly considered real-world high-dimensional tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要