一种改进的深度学习模型自适应学习率策略

An Improved Depth Learning Model Adaptive Learning Rate Strategy

针对当前深度学习中学习率不能完全拟合模型运行状态,导致收敛速度较慢和误差较大的问题,提出一种自适应学习率策略AdaDouL。该策略是在上一回合学习率的基础上利用当前的梯度去自适应调节学习率的大小,并根据损失函数的增量正负值给出2种不同下降速度的学习率形式,以模型的输出和标签之间的损失函数作为评价指标,在Vot2015数据集上使用构建的卷积神经网络模型进行实验验证。验证结果表明:使用该学习策略的深度模型相比使用AdaGrad和AadDec学习策略具有更快的收敛速度,并且收敛误差值也有一定降低;进行检测测试时,中心误差精度提升了4.5%,检测准确率上升了2.1%。

In view of the deep learning rate can not completely fit the running state model,resulting in slow convergence speed and large error,this paper proposes an adaptive learning rate strategy AdaDouL.Based on the learning rate of the last round,use the current gradient to adaptively adjust the learning rate,and according to the increment plus or minus value of the loss function,2 learning rates of different descent rates are given.With the loss of function between the model output and the label as the evaluation index,the simulation is carried out,using convolutional neural network model on Vot2015 data sets.The results show that the depth model using learning strategies has faster convergence speed than learning strategies using AdaGrad and AadDec,convergence error is reduced.The test shows that center error accuracy increased by 4.5%,the detection rate increased by 2.1%.