Cyclic Gradient Methods for Unconstrained Optimization

Gradient method is popular for solving large-scale problems. In this work, the cyclic gradient methods for quadratic function minimization are extended to general smooth unconstrained optimization problems. Combining with nonmonotonic line search, we prove its global convergence. Furthermore, the proposed algorithms have sublinear convergence rate for general convex functions, and R-linear convergence rate for strongly convex problems. Numerical experiments show that the proposed methods are effective compared to the state of the arts.