基于混合割线方程修正的L-BFGS算法

A Modified L-BFGS Algorithm Based on Hybrid Secant Equation

L-BFGS方法是解决大规模无约束优化问题最有效的拟牛顿方法之一,该方法既保持了BFGS方法在理论上良好的收敛性,又克服了拟牛顿法储存量大、计算量大的困难。大量研究表明,对割线方程进行修正能更好地逼近目标函数的二阶曲率信息,进而改善BFGS方法的计算效率。基于Li和Yuan等人提出的两种割线方程,构造了一种新的混合割线方程,并用该方程修正了L-BFGS算法,提出了一个基于混合割线方程修正的L-BFGS算法(ML-BFGS)。在适当的假设条件下,建立了ML-BFGS方法在一致凸函数上的全局收敛性,并证明了该方法是R-线性收敛的。数值结果表明,在某些情况下,ML-BFGS方法要比L-BFGS方法更优。

The L-BFGS method is one of the most effective quasi-Newton methods for solving large-scale unconstrained optimization problems.This method not only maintains the theoretically good convergence of the BFGS method,but also overcomes the difficulties of large storage of the quasi-Newton method.A large number of studies have shown that modified secant equation can better approximate the second-order curvature information of the objective function,thereby improving the calculation efficiency of the BFGS method.Therefore,based on two secant equations proposed by Li and Yuan,etc.,and a new hybrid secant equation is constructed.The L-BFGS algorithm is modified by this equation,and a modified L-BFGS algorithm based on hybrid secant equation(ML-BFGS)is proposed.Under proper assumptions,the global convergence of the ML-BFGS method on uniformly convex functions is established,and the method is proved to be R-linearly convergent.Numerical results show that in some cases,the ML-BFGS method is better than the L-BFGS method.