一种新的快速BP神经网络算法——QLMBP

QLMBP: A Quick BP Neural Network Algorithm

对反向传播(BP)算法中收敛速度最快的改进版本Levenberg-Marquardt BP(LMBP)进行了研究,找出了收敛速度的瓶颈:迭代控制参数的初始化会严重地影响到算法的迭代次数;涉及的矩阵求逆是每次迭代中最耗时的计算;如果每次迭代中的误差平方和没有变小,该次迭代可能需要很长时间.本文通过上下三角(LU)分解去除耗时的矩阵求逆,并采取一维搜索来加速目标函数值的下降,使得LMBP不再依赖于迭代控制参数,从而提出了一种快速神经网络算法QLMBP.QLMBP算法的收敛速度比LMBP算法快100倍左右.

The improved version of BP （Back Propagation） algorithm with the fastest convergence speed, LMBP （Levenberg-Marquardt BP） algorithm, is investigated, finding out the bottlenecks of the convergence speed, that is, the initialization of iteration controlling parameters has a great influence on the iterated number, the calculation of the inverse matrix involved in each iteration is the most time-consuming, and it will take long to carry out a certain interation if the sum of squared errors in each interation is not decreased. To solve these problems, LU （ Lower-Upper） decomposition is employed to avoid the time-consuming calculation of inverse matrix, and the one-dimension searching is adopted to accelerate the decrease of the object function. Thus, a quick BP neural network algorithm named QLMBP （Quick LMBP） is proposed. The proposed QLMBP algorithm is independent on the iteration controlling parameters and its convergence speed is about 100 times that of the LMBP algorithm convergence speed.