前馈神经网络新型综合算法

A new integrated algorithm of feed-forward neural network

为了实时解决前馈神经网络学习过程中可能遇到的小残量问题和大残量问题,引入QuasiNewton优化算法,并与GaussNewton法相结合,构建基于GaussNewton QuasiNewton法的前馈神经网络.根据每次迭代的结果判断属于大残量问题还是小残量问题,进而选择采取GaussNewton迭代步或QuasiNewton迭代步.与基于最速下降法的经典前馈神经网络以及与基于GaussNewton法的前馈神经网络的对比实验表明,所构造的基于Gauss Newton QuasiNewton法的前馈神经网络较好地解决了残量问题,具有良好的收敛性和稳定性.

In order to solve the little and large residual problems at the same time, QuasiNewton algorithm is introduced and combined with the GaussNewton algorithm so as to form a feed-forward neural network based on GaussNewton-QuasiNewton algorithm. According to the iteration result, little or large residual problem could be identified; and steps based on GaussNewton or QuasiNewton algorithm could be chosen as a consequence. Comparing with the feed-forward neural network based on fast descent or GaussNewton algorithm, the new feed-forward neural network based on GaussNewton-QuasiNewton algorithm can solve the residual problem properly; and its convergence and stability perform well.