基于Dopout与ADAM优化器的改进CNN算法

Modified CNN algorithm based on Dropout and ADAM optimizer

在分析当前卷积神经网络模型特征提取过程中存在问题的基础上,提出了基于Dropout与ADAM优化器的改进卷积神经网络算法（MCNN-DA）.设计了二次卷积神经网络结构,通过引入基于Re LU的激活函数以避免梯度消失问题,提高收敛速度;通过在全连接层和输出层之间加入Dropout层解决过拟合问题,并设计了ADAM优化器的最小化交叉熵.以MNIST和HCL2000数据集为测试数据,测试分析了ADAM优化器的不同学习率对算法性能的影响,得出当学习率处于0.04~0.08时,算法具有较好的识别性能.与三种算法的实验比较结果表明：本文算法的平均识别率最高可达99.21%;对于HCL2000测试集,本文算法的平均识别率比基于支持向量机优化的极速学习机算法提高了3.98%.

A modified convolution neural network（CNN） algorithm was proposed based on Dropout and adaptive moment estimation（ADAM） optimizer（MCNN-DA） by analyzing the problems of CNN in extracting the convolution feature.A quadratic convolution neural network structure was designed for MCNN-DA,and Re LU was adopted as the activation function to avoid the vanishing gradient problem and accelerate the convergence.Focusing on the over-fitting problem,the algorithm employed an ADAM optimizer to minimize the cross entropy,which was implemented by inserting a Dropout layer into the all-connected layer and the output layer.Datasets MNIST and HCL2000 were used as the benchmark data,and the performance of ADAM optimizer was analyzed under different learning parameters,which shows that the proposed algorithm has better recognition performance when the learning rate is set to 0.04~0.08.Statistic results compared with three kinds of algorithms show that for the benchmark MNIST,the MCNN-DA exhibits high recognition rate of 99.21%;compared with reduced extreme learning machine algorithm optimized with support vector machine,the proposed algorithm＇s average increase of recognition rate is 3.98% for the benchmark HCL2000.