Probabilistic Guarantees of Stochastic Recursive Gradient in Non-Convex Finite Sum Problems
CoRR(2024)
摘要
This paper develops a new dimension-free Azuma-Hoeffding type bound on
summation norm of a martingale difference sequence with random individual
bounds. With this novel result, we provide high-probability bounds for the
gradient norm estimator in the proposed algorithm Prob-SARAH, which is a
modified version of the StochAstic Recursive grAdient algoritHm (SARAH), a
state-of-art variance reduced algorithm that achieves optimal computational
complexity in expectation for the finite sum problem. The in-probability
complexity by Prob-SARAH matches the best in-expectation result up to
logarithmic factors. Empirical experiments demonstrate the superior
probabilistic performance of Prob-SARAH on real datasets compared to other
popular algorithms.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要