Steepest descent algorithms in optimization with good convergence properties
Chinese Control and Decision Conference, 2008, CCDC 2008(2008)
Abstract
There are interesting new algorithms which overcome the slow convergence near a minimum point of the standard steepest descent algorithm. For a convex quadratic function we have n-steps convergence if the step lengths are set equal to the reciprocals of the eigenvalues of the Hessian matrix. The Barzilai-Borwein step lengths are sometimes equal to the reciprocals of eigenvalues. It provides good convergence properties for quadratic functions. However with the BB-step lengths the function may increase as the iteration progresses. Also the BB-step lengths can be negative when it is applied to a nonlinear function which is nonconvex. We make some modifications in the use of the BB-step lengths to ensure monotonic decreases in the function as the iteration progresses. This leads to algorithms which have good convergence properties for a nonlinear function which can be nonconvex. We can prove global convergence for a function which has properly nested level sets by using the Inverse Lyapunov Function Theorem. ©2008 IEEE.
MoreTranslated text
Key words
barzilai-borwein step lengths,nonlinear functions,steepest descent directions,unconstrained optimization,convergence,control systems,optimal control,convex programming,algorithm design and analysis,quadratic programming,approximation algorithms,hessian matrix,level set,steepest descent,lyapunov function,eigenvalues,mathematics,nonlinear function
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined