基于冗余滤波器剪枝-辅助网络蒸馏混合压缩算法优化

Optimization of hybrid compression algorithm based on redundant filter pruning and assistant network distillation

针对师生网络差距过大、教师网络复杂度高时,学生网络对图像分类任务的准确率明显下降的问题,提出了一种基于冗余滤波器剪枝-特征相关性辅助蒸馏混合压缩算法。该算法引入了教师辅助网络,充当师生网络的媒介,有效缩小师生网络差距过大的问题;然后利用冗余滤波器结构化剪枝算法(RFSP),对教师网络和教师辅助网络进行剪枝操作;最后对剪枝后的教师网络进行中间层特征相关性蒸馏,挖掘更多教师网络的特征知识,可以更有效的将信息传递给学生网络。实验结果表明,特征相关辅助网络知识蒸馏(GW_RAKD)与RFSP算法的有效结合,能够进一步提高学生网络对图像分类的准确性。

A hybrid compression algorithm based on redundant filter pruning feature correlation assisted distillation is proposed to address the significant decrease in accuracy of image classification tasks in student networks due to the large gap between teacher and student networks and the increasing complexity of teacher networks.A teacher assisted network is introduced into this algorithm to serve as a medium for the teacher-student network,which reduce the problem of excessive network gap between teachers and students network effectively.Then,redundant filter pruning algorithm is used to prune the teacher network and teacher assisted network.Finally,the middle layer feature correlation distillation is performed on the pruned teacher network to explore more feature knowledge of the teacher network,which can more effectively transmit information to the student network.The experimental results show that the effective combination of feature related auxiliary network knowledge distillation and redundant filter pruning algorithm can further improve the accuracy of image classification of student networks.