不均衡数据对卷积神经网络的影响及改进算法

Influence upon and improved algorithm of convolutional neural networks under data imbalance

通过不同的数据分布、激活函数和网络结构对卷积神经网络(convolutional neural networks,CNN)的训练过程进行试验分析发现,数据不均衡会造成CNN训练过程收敛慢、泛化能力差的负面影响。针对这一问题,结合过抽样和欠抽样各自的优点,在随机梯度下降算法的基础上,提出均衡小批量随机梯度下降算法(equilibrium mini-batch stochastic gradient descent,EMSGD),保证小批量内的数据均衡,精确调整更新参数的梯度方向。试验结果表明,均衡小批量随机梯度下降算法可以在数据不均衡条件下提高CNN训练误差收敛速度,提高泛化性能。

Experiments under different data,activation functions and network structures show that the CNN training error converges slowly and the generalization ability is poor under imbalanced training data.In response to this problem,in combination with advantages of over-sampling and under-sampling,equilibrium mini-batch stochastic gradient descent(EMSGD)was put forward on the basis of the mini-batch stochastic gradient descent,ensuring the data balance in mini-batch and adjusting accurately the gradient direction of update parameters.Experiments prove that EMSGD can raise convergence speed of CNN training error under the condition of imbalanced training data and improve the generalization ability.