深度卷积神经网络中激活函数的研究

Research on Activation Function in Deep Convolutional Neural Network

针对深度卷积神经网络中经典的AlexNet网络模型中激活函数ReLU在网络模型训练时易产生神经元“死亡”和均值偏移的问题进行研究以及改进,通过结合反正切函数和对数函数的优势,在传统激活函数ReLU基础上提出了一种新的激活函数sArcReLU,并在后续训练过程中进一步调参。并将文中改进后的激活函数sArcReLU用于AlexNet网络模型训练,将使用新激活函数训练的深度卷积神经网络模型应用于公开数据集进行分类实验以验证其性能。实验结果表明:利用sArcReLU激活函数训练的深度卷积神经网络比利用ReLU以及ArcReLU训练的网络模型在分类精度上分别提升了1.7%和2.4%,证明了改进方式经过大量数据充分微调的深度卷积神经网络可有效地提高图像分类精度,该方法同时也提升了深度卷积神经网络的实际应用价值。

The problem that the activation function ReLU in the classic AlexNet network model of deep convolutional neural network is prone to"death"of neurons and mean shift when the network model is trained is studied and improved.By combining the advantages of arctangent function and logarithmic function,we propose a new activation function sArcReLU based on the traditional activation function ReLU and further adjust the parameters in the subsequent training process.The improved activation function sArcReLU is used for AlexNet network model training,and the deep convolutional neural network model trained with the new activation function is applied to the public data set for classification experiments to verify its performance.The experiment shows that the deep convolutional neural network trained with the sArcReLU activation function improves the classification accuracy by 1.7%and 2.4%,respectively,than the network model trained with ReLU and ArcReLU,which proves that the proposed improved method is fully fine-tuned by a large amount of data.Neural network can effectively improve the accuracy of image classification,and this method also improves the practical application value of deep convolutional neural networks.