基于自适应步长的随机递归梯度算法

Stochastic Recursive Gradient Algorithm Based on Adaptive Step Size

针对机器学习中一类有限光滑凸函数和的最小化问题,将自适应步长与SARAH++算法结合,提出了一种改进的算法SARAH++AS.然后在强凸的假设下证明了它的收敛性.最后从实验结果分析来看,相比于使用固定步长的SARAH++算法,新算法的收敛速度更快,不受初始步长选取的影响.新算法对初始步长的选择是有效的.

Aiming at the minimization problem of finite smooth convex function sum in machine learning,an improved algorithm SARAH++AS is proposed by combining adaptive step size with SARAH++algorithm.Then its convergence is proved under the strong convexity hypothesis.Finally,from the analysis of experimental results,compared with the fixed step size SARAH++algorithm,the convergence speed of the new algorithm is faster and less affected.The effect of the initial step selection.The new algorithm is effective for selecting the initial step size.