Gaussian Approximation and Multiplier Bootstrap for Polyak-Ruppert Averaged Linear Stochastic Approximation with Applications to TD Learning
CoRR(2024)
摘要
In this paper, we obtain the Berry-Esseen bound for multivariate normal
approximation for the Polyak-Ruppert averaged iterates of the linear stochastic
approximation (LSA) algorithm with decreasing step size. Our findings reveal
that the fastest rate of normal approximation is achieved when setting the most
aggressive step size α_k≍ k^-1/2. Moreover, we prove the
non-asymptotic validity of the confidence intervals for parameter estimation
with LSA based on multiplier bootstrap. This procedure updates the LSA estimate
together with a set of randomly perturbed LSA estimates upon the arrival of
subsequent observations. We illustrate our findings in the setting of temporal
difference learning with linear function approximation.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要