具有生物真实性的强抗噪性神经元激活函数

Bio-inspired Activation Function with Strong Anti-noise Ability

当前人工神经网络虽然在图像识别等方面媲美人脑,但因其所采用的激活函数ReLU和Softplus等只是对生物神经元输出响应特性的高度简化与模拟,使其在抗噪性、不确定性信息处理及功耗等方面与人脑仍存在巨大差距。通过分析生物神经元仿真实验,以其响应特性为基础,引入反映每个神经元随机性的参数η,构建出一种具有生物真实性的强抗噪性激活函数Rand Softplus。最后将该激活函数应用于深度残差网络,并基于人脸表情数据集对其进行验证。结果表明,在输入无噪声或具有少量噪声时,文中提出的激活函数与当前主流激活函数的识别精度基本持平,当输入包含较大噪声时,文中所提激活函数的识别精度远高于其他激活函数,表现出了良好的抗噪性能。

Although the artificial neural network is almost comparable to the human brain in image recognition,the activation functions such as ReLU and Softplus are only highly simplified and simulated for the output response characte-ristics of biological neurons.There is still a huge gap between the artificial neural network and the human brain in many aspects,such as noise resistance,uncertainty information processing and power consumption.In this paper,based on the simulation experiments of biological neurons and their response characteristics,a strong anti-noise activation function Rand Softplus with biological authenticity was constructed by defining and calculating parametersη,which reflects the randomness of each neuron.Finally,the activation function was applied to the depth residuals network and verified by facial expression dataset.The results show that the recognition accuracy of the activation function proposed in this paper is almost equal to the current mainstream activation function when there is no noise or a small amount of noise,and when the input contains a large amount of noise,it shows good anti-noise performance.