Inverse Free Zap Stochastic Approximation Extended Abstract

The Zap Zero algorithm is introduced, a stochastic approximation algorithm designed to approximate the Newton-Raphson flow. It is stable and convergent under under the same assumptions required for Zap stochastic approximation, which achieves the same approximation. The significant advantage of the new algorithm is that matrix inversion is avoided. To estimate a parameter of dimension d, the Zap Zero algorithm requires parameter updates of dimension 3 × d.Motivation and stability analysis is based on the ODE method and the theory of two time-scale stochastic approximation. The ODE itself can be interpreted as the solution to a particular linear quadratic regulator problem, even though the algorithm is designed for nonlinear root finding problems.