机器学习中随机递归梯度算法的步长规则

A Stochastic Recursive Gradient Algorithm by Step Size Rules in Machine Learning

针对机器学习中一类有限光滑凸函数和的最小化问题,将随机递归梯度算法和一种改进的BB步长方法有机结合,提出一种新的随机递归梯度算法.新算法的特点是对初始步长具有鲁棒性,其初始步长可以任意选取.新算法充分利用改进的BB步长方法的优势,使得算法运行过程中自适应地调节步长大小.数值实验表明新算法是可行有效的.

To propose a new modified stochastic recursive gradient algorithm for minimizing the sum of a finite number of smooth convex function,we combine the stochastic recursive gradient algorithm with an improved BB step size method.The characteristic of the algorithm is that the new algorithm is robust on the initial step size which can be arbitrarily selected and automatically adjust in the running process of the algorithm.Numerical experiments indicate that the proposed algorithm is feasible and effective.