面向嵌入式应用的深度神经网络压缩方法研究

Compression Methods of Deep Neural Network for Embedded Systems

深度神经网络近几年在图像处理、目标识别等应用中取得了巨大的成功,但深度神经网络过多的参数使其计算成本高且存储耗费大,很难部署在嵌入式设备等资源受限的硬件平台上。为解决该问题,采用矩阵奇异值分解(SVD)和网络剪枝两种方法压缩深度神经网络,并分析两种压缩方法在不同的硬件条件下的适用性。SVD方法通过引入神经元数更少的中间层降低权重规模和连接数;网络剪枝方法先剪去网络中权重小于某一阈值的连接,再重新训练稀疏连接的网络。在基本不损失精度的前提下,这两种方法对改进的深度神经网络PVANet分别压缩了5×、10×。SVD策略和网络剪枝方法为深度神经网络的嵌入式应用提供了可行的解决方案。

In recent years,deep neural networks have achieved great success in a wide range of applications including image processing,object recognition,etc.However,the large number of parameters of deep network models consume high computation cost and considerable storage requirement,making it difficult to deploy these deep networks on resource constrained hardware platforms such as embedded devices.To solve this problem,matrix singular value decomposition（SVD）and network pruning were adopted to compress deep neural networks and then the applicability of the two compression methods under different hardware conditions was analyzed.The SVD method reduced the number of weights by introducing an intermediate layer with fewer neurons.The network pruning method first pruned unimportant connections whose weights were smaller than a certain threshold and then retrained the sparsely connected network.In the experiment,the improved deepneural network PVANet was basically compressed by 5× and 10× respectively by these two methods without accuracy loss.The SVD and network pruning methods provide feasible solutions for embedded applications of deep neural networks.