卷积神经网络中激活函数的性质分析与改进

Analysis and Improvement of Properties of Activation Functions in Convolutional Neural Networks

为了提高卷积神经网络模型的效率,针对激活函数进行了研究。通过研究多种激活函数的发展进程,列举各类激活函数的性质,以及在分析模型前向传播和反向传播中激活函数所起到的作用的基础上,提出了改进的激活函数Re LU-Xe X,有效的缓解了梯度消失、“神经元坏死”等问题。在MNIST、CIFAR-10、CIFAR-100、MSTAR数据集上的试验结果表明,改进的激活函数Re LU-Xe X整体表现强于其它的一些激活函数,且在模型的收敛速度上有所提升。对激活函数的性质进行了较为深入的研究,将激活函数的理论和实际表现进行了对照,用算法仿真论证了改进的激活函数Re LU-Xe X在理论上存在的优势。

In order to improve the efficiency of the convolutional neural network model, this paper studied the activation function. By studying the development process of various activation functions, enumerating the properties of various activation functions and analyzing the role of activation functions in forward and back propagation of the model, an improved activation function Re LU-Xe X is proposed, which effectively alleviated the problems of gradient disappearance and " necrosis of neurons". The experimental results on the MNIST, CIFAR-10, CIFAR-100, and MSTAR datasets show that the improved activation function Re LU-Xe X overall performance is stronger than some other activation functions, and the convergence speed of the model has been improved. In this paper, a more indepth study of the nature of the activation function was carried out, the theoretical and actual performance of the activation function was compared, and the theoretical advantages of the improved activation function Re LU-Xe X were demonstrated by algorithm simulation experiments.