一种基于改进Softplus激活函数的卷积神经网络模型

A convolutional neural network model based on improved Softplus activation function

为提高Softplus函数在神经网络中的性能,针对Softplus函数的缺点提出了一种基于改进Softplus激活函数的卷积神经网络模型。根据“输出均值接近零的激活函数可以提升神经网络的学习性能”原理,首先对原函数的输出向下平移缓解Softplus激活函数的均值偏移现象;然后对调整后的输出乘以一个参数调整函数在正数部份的坡度和负数部分的饱和点位置,以缓解在训练过程中出现的梯度消失/溢出现象。最后在MNIST、CIFAR-10和CIFAR-100数据集上的实验结果表明,同其它常用激活函数相比,改进后的Softplus函数取得了较好的识别率。

To solve this problem,this paper proposes a convolutional neural network model based on the improved Softplus activation function.According to the principle that"the activation function whose average output is close to zero can improve the learning performance of the neural network",the output value of the original function is shifted downward to alleviate the offset phenomenon of the Softplus activation function firstly.Then the offset function output is multiplied by a parameter to adjust the slope and the saturation point of the function.This is used to alleviate the gradient disappearance/overflow phenomenon that occurs during training.Finally,the experimental results on the MNIST,CIFAR-10 and CIFAR-100 datasets show that the improved Softplus function achieves a better recognition rate than other commonly used activation functions.