二阶锥规划的一种Barzilai-Borwein梯度算法

二阶锥规划是一个非光滑的凸规划问题,在电子工程、控制论、组合优化等许多工程问题中有着广泛的应用。提出了一种求解二阶锥规划的Barzilai-Borwein梯度算法。首先基于二阶锥规划的最优性条件,二阶锥规划问题被等价转化为等式系统。然后利用一种有效的光滑技巧把二阶锥转化为光滑函数,该等式系统等价转化为一个光滑的无约束优化问题。最后利用Barzilai-Borwein梯度法求解该无约束优化问题。Barzilai-Borwein梯度法是一种有效的一阶梯度算法,理论证明了该算法的收敛性。通过4个仿真算例测试提出的算法,实验结果表明了该算法有着较高的精度,需要较少的计算时间,所以该算法是可行的和有效的。

A Mini-Batch Proximal Stochastic Recursive Gradient Algorithm with Diagonal Barzilai–Borwein Stepsize

Many machine learning problems can be formulated as minimizing the sum of a function and a non-smooth regularization term.Proximal stochastic gradient methods are popular for solving such composite optimization problems.We propose a minibatch proximal stochastic recursive gradient algorithm SRG-DBB,which incorporates the diagonal Barzilai–Borwein(DBB)stepsize strategy to capture the local geometry of the problem.The linear convergence and complexity of SRG-DBB are analyzed for strongly convex functions.We further establish the linear convergence of SRGDBB under the non-strong convexity condition.Moreover,it is proved that SRG-DBB converges sublinearly in the convex case.Numerical experiments on standard data sets indicate that the performance of SRG-DBB is better than or comparable to the proximal stochastic recursive gradient algorithm with best-tuned scalar stepsizes or BB stepsizes.Furthermore,SRG-DBB is superior to some advanced mini-batch proximal stochastic gradient methods.

NONMONOTONE LOCAL MINIMAX METHODS FOR FINDING MULTIPLE SADDLE POINTS

In this paper,by designing a normalized nonmonotone search strategy with the BarzilaiBorwein-type step-size,a novel local minimax method(LMM),which is a globally convergent iterative method,is proposed and analyzed to find multiple(unstable)saddle points of nonconvex functionals in Hilbert spaces.Compared to traditional LMMs with monotone search strategies,this approach,which does not require strict decrease of the objective functional value at each iterative step,is observed to converge faster with less computations.Firstly,based on a normalized iterative scheme coupled with a local peak selection that pulls the iterative point back onto the solution submanifold,by generalizing the Zhang-Hager(ZH)search strategy in the optimization theory to the LMM framework,a kind of normalized ZH-type nonmonotone step-size search strategy is introduced,and then a novel nonmonotone LMM is constructed.Its feasibility and global convergence results are rigorously carried out under the relaxation of the monotonicity for the functional at the iterative sequences.Secondly,in order to speed up the convergence of the nonmonotone LMM,a globally convergent Barzilai-Borwein-type LMM(GBBLMM)is presented by explicitly constructing the Barzilai-Borwein-type step-size as a trial step-size of the normalized ZH-type nonmonotone step-size search strategy in each iteration.Finally,the GBBLMM algorithm is implemented to find multiple unstable solutions of two classes of semilinear elliptic boundary value problems with variational structures:one is the semilinear elliptic equations with the homogeneous Dirichlet boundary condition and another is the linear elliptic equations with semilinear Neumann boundary conditions.Extensive numerical results indicate that our approach is very effective and speeds up the LMMs significantly.