随机加速梯度算法的回归学习收敛速度

On Stochastic Accelerated Gradient with Convergence Rate of Regression Learning

该文考虑两个经典监督学习问题(即最小二乘和logistic回归)的随机逼近.在损失函数假设非强凸性基础上,减弱了梯度的Lipschitz连续条件,提出了两种加速随机梯度算法.通过对大多数现有工作中的经验风险(期望)的非渐近分析,得到该算法的收敛速度为O(1/n),其中n是样本数量.与已知的结果相比,只需要较少的条件就可以得到最小二乘回归和logistic回归问题的收敛速度.

This paper studies the regression learning problem from given sample data by using stochastic approximation(SA)type algorithm,namely,the accelerated SA.We focus on problems without strong convexity,for which all well known algorithms achieve a convergence rate for function values of O(1/n).We consider and analyze accelerated SA algorithm that achieves a rate of O(1/n)for classical least square regression and logistic regression problems respectively.Comparing with the well known results,we only need fewer conditions to obtain the tight convergence rate for least square regression and logistic regression problems.