多层前向神经网络的新型二阶学习算法

New Second Order Learning Algorithm of Multilayer Feedforward Neural Network

提出了多层前向神经网络的新型二阶递推学习算法 .该算法不仅能使网络各层误差而且使二阶导数信息因子反向传播 .证明了新算法等价于Newton迭代法并且有二阶收敛速度 .它实现了Newton搜索方向和Hessian阵逆的递推运算 ,其计算量几乎与普通递推最小二乘法相当 .由算法性能分析证明新算法优于Karayiannis等人的二阶学习算法 .

A new second order recursive learning algorithm to multilayer feedforward network is proposed.This algorithm makes not only each layer errors of network but also second order derivative information factors backpropagate.And it is proved that it is equivalent to Newton iterative algorithm and has second order convergent speed.New algorithm achieves the recurrence calculation of Newton search directions and the inverse of Hessian matrices.Its calculation quantity is correspond to that of common recursive least squares algorithm.It is stated clearly that this new algorithm is superior to Karayiannis second order algorithm according to analysis of their properties.