基于曲率信息的改进BP算法及其在FNN中的应用

Curvature-based BP Algorithm Optimization and Its Application in FNN

针对步长选取影响误差反向传播(BP,Back Propagation)算法优化效率问题,提出一种基于曲率信息的步长优化BP算法,并将其应用到了模糊神经网络(FNN)的训练过程中。参考牛顿法的思想,根据代价函数的梯度及梯度方向上的曲率信息来确定模型参数调整的方向和幅度。仅需考虑梯度方向上的二阶信息,因此不需要存储和处理Hessian矩阵。通过一个数值仿真和高炉炼铁过程数据建模实验,验证了方法的有效性及训练效率。

In order to improve the optimization efficiency of BP algorithm affected by the selection of step size, a step size optimization BP algorithm based on curvature information is proposed and applied to the training process of FNN(Fuzzy Neural Network). Reference to Newton’s method, The gradient of the cost function and the curvature information in the direction are calculated to determine the direction and magnitude of the parameter adjustment in each iteration. This method only needs to consider the two order information of the gradient direction, so it does not need the storage and processing of Hessian matrix. The effectiveness and efficiency of the proposed method are verified by a numerical simulation and data simulation of blast furnace ironmaking process.