深度神经网络模型压缩综述

Survey of Deep Neural Networks Model Compression

近年来,随着深度学习的飞速发展,深度神经网络受到了越来越多的关注,在许多应用领域取得了显著效果。通常,在较高的计算量下,深度神经网络的学习能力随着网络层深度的增加而不断提高,因此深度神经网络在大型数据集上的表现非常卓越。然而,由于其计算量大、存储成本高、模型复杂等特性,使得深度学习无法有效地应用于轻量级移动便携设备。因此,压缩、优化深度学习模型成为目前研究的热点。当前主要的模型压缩方法有模型裁剪、轻量级网络设计、知识蒸馏、量化、体系结构搜索等。对以上方法的性能、优缺点和最新研究成果进行了分析总结,并对未来研究方向进行了展望。

In recent years,the deep neural networks have gained more and more attention with the rapid development of deep learning.It has achieved remarkable effect in many application fields.Usually,at a higher computation,the learning ability of deep neural networks is improved with the increase of depth,which makes the performance of deep learning on large datasets especially successful.However,the deep learning can􀆳t be effectively applied to the lightweight mobile portable device due to the characteristics of large amount of calculation,high storage cost and complicated model.Therefore,compressing and simplifying the deep learning model has become the research hot spot.Currently,the main model compression methods include pruning,lightweight network design,knowledge distillation,quantization,neural architecture search,etc.This paper analyses and summarizes the performance,advantages and limitations and the latest research results of the model compression methods,and prospects the future research direction.