一种基于综合目标函数的神经网络学习算法

A novel neural network training algorithm based on generalized objective function

为提高多层前向神经网络的学习速度和算法的稳定性,提出一种基于综合目标函数的改进学习算法。该算法在误差平方和目标函数中引入一个辅助约束项构成综合目标函数,并利用综合目标函数训练网络的输出层权值,采用牛顿法推导出训练输出层权值的递推公式。辅助约束项隐含有对网络输出平滑性的约束,提高了学习算法的稳定性。利用该算法对不同非线性函数生成的样本数据的学习结果表明,新算法的收敛速度、精度均优于Karayiann is等人的二阶学习算法。

A novel training algorithm was proposed to improve the learning rate and stability of the multi-layer feedforward neural networks. The generalized objective function was constructed by adding an auxiliary constraint term to the sum of the squared errors in the algorithm. The weight matrix of output layer was trained using the generalized objective function. The recursive equations for training the weight matrix of output layer were derived using Newton iterative algorithm without any simplification. The auxiliary constraint term involves the requirement for the smoothness of output which could improve the stability of the algorithm. The high-order derivative information of the neuron action function was used during the training procedure, so the algorithm had high convergence speed. In the end, the algorithm was used to learn training pattern of different nonlinear function. Simulation results show that the convergent rate and accuracy of the algorithm are better than those of the Karayiannis＇s second-order learning algorithm.