The Power of Extrapolation in Federated Learning
arxiv(2024)
摘要
We propose and study several server-extrapolation strategies for enhancing
the theoretical and empirical convergence properties of the popular federated
learning optimizer FedProx [Li et al., 2020]. While it has long been known that
some form of extrapolation can help in the practice of FL, only a handful of
works provide any theoretical guarantees. The phenomenon seems elusive, and our
current theoretical understanding remains severely incomplete. In our work, we
focus on smooth convex or strongly convex problems in the interpolation regime.
In particular, we propose Extrapolated FedProx (FedExProx), and study three
extrapolation strategies: a constant strategy (depending on various smoothness
parameters and the number of participating devices), and two
smoothness-adaptive strategies; one based on the notion of gradient diversity
(FedExProx-GraDS), and the other one based on the stochastic Polyak stepsize
(FedExProx-StoPS). Our theory is corroborated with carefully constructed
numerical experiments.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要