基于网络特征的分层剪枝方法

Layer-Wise Pruning Method Based on Network Characteristics

针对传统分层剪枝方法在剪枝过程后期时,网络模型的准确率会随网络结构失衡陡然下降的问题,提出一种基于网络特征的分层剪枝方法.该方法首先根据网络深度、网络宽度、层间重要性指标计算每轮迭代的剪枝系数;然后结合基础剪枝率得到每层参数的动态剪枝率;最后对预训练的网络进行剪枝、微调,并重复上述过程至迭代结束.实验结果表明,基于网络特征的分层剪枝方法在VGG-16模型上表现良好,在压缩率提高约一倍的情况下,准确率仍比单剪枝率的分层剪枝方法高3.6%,且整体表现优于全局剪枝方法.当压缩率达到98.85%以上时,在Resnet-20模型上的准确率比单剪枝率的分层方法高20%,接近于全局剪枝方法,表明充分利用网络特征可提高分层剪枝方法的性能.

Aiming at the problem that the accuracy of the network model would drop sharply with the imbalance of the network structure when the traditional layer-wise pruning method was in the later stage of the pruning process, we proposed a layer-wise pruning method based on network characteristics. Firstly, the pruning coefficient of each iteration was calculated according to the network depth, network width and inter-layer importance index. Secondly, the dynamic pruning rate of each layer parameter was obtained by combining the basic pruning rate. Finally, pruned and fine tuned the pre-trained network, and repeated the above process to the end of iteration. The experimental results show that the layer-wise pruning method based on network characteristics performs well on VGG-16 model. When the compression rate is about double, the accuracy is still 3.6% higher than that of the layer-wise pruning method with single pruning rate, and the overall performance is better than that of the global pruning method. When the compression rate reaches more than 98.85%, the accuracy on the Resnet-20 model is 20% higher than that of the layer-wise method with single pruning rate, which is close to the global pruning method. It shows that the performance of layer-wise pruning method can be improved by making full use of network characteristics.