Lifted Primal-Dual Method for Bilinearly Coupled Smooth Minimax Optimization

We study the bilinearly coupled minimax problem: min(x) max(y) f (x) + < y, Ax > - h(y), where f and h are both strongly convex smooth functions and admit first-order gradient oracles. Surprisingly, no known first-order algorithms have hitherto achieved the lower complexity bound of Omega((root Lx/mu x + parallel to A parallel to/root mu(x)mu(y) + root L-y/mu(y)) log(1/epsilon)) for solving this problem up to an epsilon primal-dual gap in the general parameter regime, where L-x, L-y, mu(x), mu(y) are the corresponding smoothness and strongly convexity constants. We close this gap by devising the first optimal algorithm, the Lifted Primal-Dual (LPD) method. Our method lifts the objective into an extended form that allows both the smooth terms and the bilinear term to be handled optimally and seamlessly with the same primal-dual framework. Besides optimality, our method yields a desirably simple single-loop algorithm that uses only one gradient oracle call per iteration. Moreover, when f is just convex, the same algorithm applied to a smoothed objective achieves the nearly optimal iteration complexity. We also provide a direct single-loop algorithm, using the LPD method, that achieves the iteration complexity of O(root L-x/epsilon + parallel to A parallel to/root mu(y)epsilon + root L-y/epsilon). Numerical experiments on quadratic minimax problems and policy evaluation problems further demonstrate the fast convergence of our algorithm in practice.