基于双曲正切函数的修正线性单元

Tangent-based rectified linear unit

为解决修正线性单元(ReLU)的神经元死亡现象,提出一个新的激活函数——基于双曲正切函数(tanh)的修正线性单元(ThLU)。ThLU函数正半轴来自于ReLU函数的正半轴,负半轴来自于tanh函数的负半轴。为验证ThLU函数的性能,基于VggNet-16神经网络架构,分别在CIFAR-10和CIFAR-100数据集上进行了试验验证。结果表明:基于ThLU函数训练得到的神经网络模型比基于tanh、ReLU、泄露修正线性单元(LReLU)和指数线性单元(ELU)训练得到的神经网络模型具有更高的准确率、更低的损失。

To solve neuron death problemofrectified linear unit(ReLU),a new activation function:tangent-based rectified linear unit(ThLU)was proposed.The positive half axis of ThLU came from the positive half axis ofReLU.And the negative half axis of ThLU came from the negative half axis of hyperbolic tangent(tanh).To verify the performance of ThLU,experiments based on the VggNet-16 neural network architecture,and the CIFAR-10 and CIFAR-100 datasets were designed,which indicated that the neural network model based on ThLU obtained higher accuracy and lower loss thanthat based on tanh,ReLU,leaky rectified linear unit(LReLU)and exponential linear unit(ELU).