Ridge Polynomial神经网络带动量项异步梯度算法的收敛性

Convergence of Asynchronous Gradient Method with Momentum for Ridge Polynomial Neural Networks

将动量项引入到Ridge Polynomial神经网络异步梯度训练算法的误差函数中,有效地改善了算法的收敛效率,并从理论上分析了Ridge Polynomial神经网络的带动量项的异步梯度算法的收敛性,给出了算法的单调性和收敛性(包括强收敛性和弱收敛性)。算法的这些收敛性质对于如何选取学习率和初始权值来进行高效的网络训练是非常重要的。最后通过计算机仿真实验验证了带动量项的异步梯度算法的高效性和理论分析的正确性。

The momentum was introduced into the conventional error function of asynchronous gradient method to im- prove the convergence efficiency of Ridge Polynomial neural network. This paper studied the convergence of the asyn- chronous gradient method with momentum for training Ridge Polynomial neural network, and a monotonicity theorem and two convergence theorems were proved, which are important for choosing appropriate learning rate and initial weights to perform an effective training. To illustrate above theoretical finding, a simulation experiment was presented.