基于Adam优化的卷积神经网络随机共振现象研究

Research on Stochastic Resonance of Convolutional Neural Network Based on Adam Optimization

文中利用随机共振对改善Adam优化的卷积神经网络在算力有限下的性能进行了研究。对反向传播算法采用动量梯度下降算法去更新参数的网络进行Adam优化,利用MNIST手写数字集进行仿真实验,在论文实验条件下,跟动量梯度下降算法的网络相比,Adam优化的网络在前15次epoch下交叉熵降低,增加训练样本数量可以使得交叉熵的降低幅度减小。论文对Adam优化的卷积神经网络的输出神经元加入高斯噪声,仿真结果表明,交叉熵减少百分比出现了随机共振现象,增加训练样本数量可以使得随机共振现象的效果减小。

In this paper,stochastic resonance is used to improve the performance of Adam optimized convolutional neural net⁃work under limited computational force.For the back-propagation algorithm,the momentum gradient descent algorithm is used to update the network parameters for Adam optimization,and the MNIST handwritten numeral set is used for simulation experiments.Under the experimental conditions of this paper,compared with the network of momentum gradient descent algorithm,the crossover entropy of Adam optimized network is reduced under the first 15 epochs.Increasing the number of training samples can reduce the reduction of cross entropy.This paper adds Gaussian noise to the output neurons of the convolution neural network optimized by Ad⁃am.The simulation results show that the stochastic resonance phenomenon occurs in the percentage of cross entropy reduction.In⁃creasing the number of training samples can reduce the effect of stochastic resonance phenomenon.