基于因子分析的卷积神经网络模型压缩算法研究

Convolutional Neural Network Model Compression Algorithm Based on Factor Analysis

针对复杂的卷积神经网络模型存在参数规模大、运算时间长等问题,提出一种有效的卷积神经网络模型压缩算法.该算法引入因子分析的思想对卷积神经网络模型进行压缩:首先将四维的卷积核权重张量转化为二维的矩阵形式,计算相关矩阵,并对其进行奇异值分解;其次,通过控制累积方差贡献率,确定适当的因子数量,计算因子载荷矩阵;最后,重构出更具代表性的卷积核.通过在Catdog、CIFAR10、CIFAR100三个数据集上进行验证,实验结果表明:该压缩算法能够在保证卷积神经网络精度的前提下,使AlexNet、ResNet的参数压缩率达到30.7%~68.2%,运行时间减少17.53%~37.21%.从而验证了本文提出的算法在压缩率和运算效率方面的优势,为基于因子分析的卷积神经网络模型压缩提供了一种可能的框架.

Aiming at the problems of large parameter scale and long operation time of complex convolutional neural network models,an effective convolutional neural network model compression algorithm was proposed.The factor analysis was introduced to compress the convolutional neural network in this algorithm.Firstly,the four-dimensional weight tensor of the convolutional kernel was transformed into a two-dimensional matrix.The correlation matrix was calculated and the singular value decomposition was performed.Secondly,the appropriate number of factors and the factor load matrix were determined by controlling the variance contribution rate.Finally,a more representative convolution kernel was reconstructed.Through the verification on three data sets of Catdog,CIFAR10 and CIFAR100,the experimental results show that the compression rate of AlexNet and ResNet parameters can reach 30.7%-68.2%,and the running time can be reduced by 17.53%-37.21%under ensuring the accuracy of convolutional neural network.Thus,the advantages of the proposed algorithm in the compression rate and operational efficiency are verified.A possible framework was provided for the model compression of convolutional neural networks based on factor analysis.