摘要
针对步长选取影响误差反向传播(BP,Back Propagation)算法优化效率问题,提出一种基于曲率信息的步长优化BP算法,并将其应用到了模糊神经网络(FNN)的训练过程中。参考牛顿法的思想,根据代价函数的梯度及梯度方向上的曲率信息来确定模型参数调整的方向和幅度。仅需考虑梯度方向上的二阶信息,因此不需要存储和处理Hessian矩阵。通过一个数值仿真和高炉炼铁过程数据建模实验,验证了方法的有效性及训练效率。
In order to improve the optimization efficiency of BP algorithm affected by the selection of step size, a step size optimization BP algorithm based on curvature information is proposed and applied to the training process of FNN(Fuzzy Neural Network). Reference to Newton’s method, The gradient of the cost function and the curvature information in the direction are calculated to determine the direction and magnitude of the parameter adjustment in each iteration. This method only needs to consider the two order information of the gradient direction, so it does not need the storage and processing of Hessian matrix. The effectiveness and efficiency of the proposed method are verified by a numerical simulation and data simulation of blast furnace ironmaking process.
作者
熊伟丽
孙文心
史旭东
Xiong Weili;Sun Wenxin;ShiXudong(School of Internet of Things Engineering,Jiangnan University,Wuxi 214122,China;Key Laboratory of Advanced Process Control for Light Industry(Ministry of Education),Jiangnan University,Wuxi 214122,China)
出处
《系统仿真学报》
CAS
CSCD
北大核心
2020年第1期1-8,共8页
Journal of System Simulation
基金
国家自然科学基金(61773182)
关键词
步长优化
BP算法
模糊神经网络
学习速度
Step size optimization
BP algorithm
Fuzzy neural network
Learning speed