前馈型神经网络新学习算法的研究

Research of new learning method of feedforward neural network

前馈神经网络在非线性系统的建模及控制中有着广阔的应用前景，但是该网络的学习算法—向后传播算法（Ｂａｃｋｐｒｏｐａｇａｔｉｏｎ（ＢＰ）Ａｌｇｏｒｉｔｈｍ）算法存在一些不足。为了提高多层前馈神经网络的学习效率及稳定性，引入了非线性最小二乘法。通过与其他学习算法的比较，得出结论：其中用差商近似代替导数的Ｐｏｗｅｌ法是一种高效、快速的学习方法，其学习速率比带动量项的学习率自适应的ＢＰ算法高一个量级，而比ＤａｖｉｄｅｎＦｌｅｔｃｈｅｒＰｏｗｅｌ（ＤＦＰ）、ＢｒｏｙｄｅｎＦｌｅｔｃｈｅｒＧｏｌｄｆａｒｌＳｈａｎｎｏ（ＢＦＧＳ）等变尺度方法以及其他非线性最小二乘法的稳定性要好得多。

Feedforward neural network has a wide applied prospect in the modeling and control of nonlinear system, but the learning method of the network Backpropagation (BP) algorithm has some shortcomings. In order to improve the learning efficiency and stability of multi layer feedforward neural network, the least square method is introduced. Through the comparison with other learning methods, the conclusion is drawn that the Powell method in which the derivative is replaced by the difference quotient is an efficient and fast method. The learning speed of the Powell method is faster than the adapted learning rate BP method with momentum and the stability is much better than variable metric method such as Daviden Flecher Powell (DFP) and Broyden Flecher Goldfarl Shanno (BFGS) and other nonlinear least square method.