Analysis of Kernel Mirror Prox for Measure Optimization
International Conference on Artificial Intelligence and Statistics(2024)
摘要
By choosing a suitable function space as the dual to the non-negative measure
cone, we study in a unified framework a class of functional saddle-point
optimization problems, which we term the Mixed Functional Nash Equilibrium
(MFNE), that underlies several existing machine learning algorithms, such as
implicit generative models, distributionally robust optimization (DRO), and
Wasserstein barycenters. We model the saddle-point optimization dynamics as an
interacting Fisher-Rao-RKHS gradient flow when the function space is chosen as
a reproducing kernel Hilbert space (RKHS). As a discrete time counterpart, we
propose a primal-dual kernel mirror prox (KMP) algorithm, which uses a dual
step in the RKHS, and a primal entropic mirror prox step. We then provide a
unified convergence analysis of KMP in an infinite-dimensional setting for this
class of MFNE problems, which establishes a convergence rate of O(1/N) in the
deterministic case and O(1/√(N)) in the stochastic case, where N is the
iteration counter. As a case study, we apply our analysis to DRO, providing
algorithmic guarantees for DRO robustness and convergence.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要