基于信息融合策略的卷积神经网络剪枝方法

CNN Pruning Method Based on Information Fusion Strategy

针对现有结构化剪枝方法过度依赖预训练模型和直接丢弃不重要卷积核的信息造成了明显的性能下降的问题,提出一种基于信息融合策略的卷积神经网络剪枝方法(APBM),以较小精度损失降低模型复杂度、实现模型加速。首先APBM方法引入信息熵概念以表示卷积核的相似度分布,并使用分布之间的相对熵动态衡量卷积核的重要程度;同时在训练的前向传播中采用信息融合策略:融合非重要卷积核信息与重要卷积核信息,以减少剪枝过程中的信息损失和提高剪枝的容错性。在CIFAR10和CIFAR100数据集上进行验证和对比实验。实验结果表明:相对于HRank、Polarization、SWP等剪枝算法,APBM方法训练时间更少、模型压缩率更高,精度保持最佳。在基于CIFAR10的剪枝任务中,对VGG16和ResNet56分别剪掉92.74%和48.84%的参数量;在基于CIFAR100的剪枝任务中,对VGG16和ResNet56分别剪掉72.91%和44.18%的参数量。

Current structured pruning methods rely too much on the pre-training models and directly discard unimportant convolution kernel information,resulting in performance degradation that cannot be ignored.Aiming at the problems,a convolution neural network pruning method(APBM)is proposed to reduce model complexity and achieve model acceleration.The APBM method introduces the concept of information entropy to represent the similarity distribution of the convolution kernels,then uses the relative entropy between the distributions to dynamically measure the importance of the convolution kernels.At the same time,in order to reduce the information loss in the pruning process and improve the fault tolerance of the method,the information fusion strategy,fusing the unimportant convolution kernels information and the important convolution kernels,is adopted in the forward propagation of model training.Experiments are carried out on the CIFAR-10 and CIFAR-100 datasets.The experimental results show that the APBM method achieves less training time,higher model compression rate,and best accuracy compared to the network pruning algorithms such as HRank,Polarization,and SWP.On CIFAR-10,the method effectively reduces the parameters of VGG16 and ResNet56 by 92.74%and 48.84%respectively without significant decrease in accuracy.On CIFAR-100,this method reduces the parameters of VGG16 and ResNet56 by 72.91%and 44.18%respectively with a little accuracy loss.