深度学习中三种常用激活函数的性能对比研究

A Comparative Study on the Performance of three kinds of Activation Functions in Deep Learning

以目前神经网络研究领域应用比较广泛的Sigmoid函数、双曲正切函数以及校正激活函数为研究对象,通过谷歌发布的机器学习平台Tensorflow中的神经网络可视化实验操作平台Playground,在给定网络拓扑结构的情况下,对三种激活函数的性能展开对比研究。分别在调整学习率与网络深度的情况下观察基于上述三种激活函数的神经网络的分类效果。通过分析实验结果,探讨三种激活函数的应用场景,并对深度神经网络中激活函数的选择方式给出建议。

Put the S function,hyperbolic tangent function and Rectified Linear Units function which are widely used in the research field of the neural network as the research object.The performance of three kinds of activation functions is compared in the case of a given network topology through the Playground in Tensorflow,an experimental platform for visualization of neural networks published by Google.The classification results of neural networks based on the above three activation functions were observed under the condition of adjusting the learning rate and the depth of the network respectively.By analyzing the experimental data,the application scenarios of the three kinds of activation functions are discussed and the selection of the activation function in the depth neural network is given.