深度网络模型压缩综述

Survey of Deep Neural Network Model Compression

深度网络近年来在计算机视觉任务上不断刷新传统模型的性能,已逐渐成为研究热点.深度模型尽管性能强大,然而由于参数数量庞大、存储和计算代价高,依然难以部署在受限的硬件平台上(如移动设备).模型的参数在一定程度上能够表达其复杂性,相关研究表明,并不是所有的参数都在模型中发挥作用,部分参数作用有限、表达冗余,甚至会降低模型的性能.首先,对国内外学者在深度模型压缩上取得的成果进行了分类整理,依此归纳了基于网络剪枝、网络精馏和网络分解的方法;随后,总结了相关方法在多种公开深度模型上的压缩效果;最后,对未来的研究可能的方向和挑战进行了展望.

Deep neural networks have continually surpassed traditional methods on a variety of computer vision tasks. Though deep neural networks are very powerful, the large number of weights consumes considerable storage and calculation time, making it hard to deploy on resource-constrained hardware platforms such as mobile system. The number of weights in deep neural networks represents the complexity to an extent, but not all the weights contribute to the performance according to recent researches. Specifically, some weights are redundant and even decrease the performance. This survey offers a systematic summarization of existing research achievements of the domestic and foreign researchers in recent years in the aspects of network pruning, network distillation, and network decomposition. Furthermore, comparisons of compression performance are provided on several public deep neural networks. Finally, a perspective of future work and challenges in this research area are discussed.