BN对VGG神经网络的影响研究

Research on the effect of batch normalization on VGG-like neural networks

文章针对在训练目前卷积神经网络中较为主流的深度神经网络VGG网络模型时调参艰难、收敛较慢的问题,引入批归一化(batch normalization,BN)进行改进。批归一化能提高网络训练的初始学习率上限,同时加快模型收敛速度。相关实验结果表明,在端对端训练或者微调神经网络过程中应用批归一化,能较好地达到优化目的,同时指出在VGG网络中所有激活层前进行批归一化能得到最好的效果。另外VGG网络的优化方法会影响到批归一化,使用改进的基于动量的随机梯度下降能使网络训练时的波动更小。

The possibility of training deep VGG-like neural networks using batch normalization(BN)to solve the problems such as the difficulty in parameter tuning and the slow convergence is analyzed.BN can raise the limit of initial learning rate and accelerate the convergence.With the theoretical analysis and experiments,it is shown that applying BN in the process of end-to-end training or fine-tuning neural networks can get better convergence effectiveness,while applying BN before all activation layers in VGG network is the best way.In addition,the optimization method of VGG network affects the convergence of network training.Using the improved momentum-based stochastic gradient descent can make the fluctuation of the training smaller.