用于图像分类的模糊策略学习率ResNet

Fuzzy policy learning rate ResNet for image classification

ResNet深度神经网络用于图像分类时,全连接层训练算法收敛性差降低了分类效果。针对此不足,提出一种模糊策略梯度算法训练ResNet。推导出ResNet全连接层权重的迭代公式,用历史梯度信息修正当前一阶小批量梯度,用模糊策略学习率更新权重,通过上下边界函数处理学习率的过大或过小而引发的迭代振荡,改善训练算法收敛性。在CINIC-10和CIFAR-100数据集上的实验结果表明,所提算法训练的ResNet分类效果优于相比较算法。特别是在综合性分类指标Kappa系数上,所提算法训练的ResNet较最新的AdaBound算法平均提高了9.29%,改进效果显著。

When the ResNet depth network is used for image classification,the poor convergence of full connection layer trai-ning algorithm reduces the classification effect.To solve this problem,a fuzzy strategy gradient algorithm was proposed to train ResNet.The iterative formula of ResNet full connection layer weight was derived,the current first-order small batch gradient was modified with the historical gradient information,the weight was updated with the fuzzy strategy learning rate,and the upper and lower boundary functions were used to deal with the iterative oscillation caused by the high or low learning rate,which improved the convergence of the training algorithm.Experimental results on CINIC-10 and CIFAR-100 datasets show that the ResNet classification effect trained using the proposed algorithm is better than that using the comparative algorithms.Especially on the comprehensive classification index Kappa coefficient,the ResNet trained using the proposed algorithm is 9.29%higher than that using the latest AdaBound algorithm on average,its improvement effect is significant.