摘要
近年来深度神经网络在目标识别、图像分类等领域取得了重大突破,然而训练和测试这些大型深度神经网络存在几点限制:1)训练和测试这些深度神经网络需要进行大量的计算(训练和测试将消耗大量的时间),需要高性能的计算设备(例如GPU)来加快训练和测试速度;2)深度神经网络模型通常包含大量的参数,需要大容量的高速内存来存储模型。上述限制阻碍了神经网络等技术的广泛应用(现阶段神经网络的训练和测试通常是在高性能服务器或者集群下面运行,在一些对实时性要求较高的移动设备(如手机)上的应用受到限制)。文中对近年来的压缩神经网络算法进行了综述,系统地介绍了深度神经网络压缩的主要方法,如裁剪方法、稀疏正则化方法、分解方法、共享参数方法、掩码加速方法、离散余弦变换方法,最后对未来深度神经网络压缩的研究方向进行了展望。
In recent years,deep neural networks have achieved significant breakthroughs in target recognition,image classification,etc.However, training and testing for these deep neural network have several limitations.Firstly,training and testing for these deep neural networks require a lot of computation (training and testing consume a lot of time),which requires high-performance computing devices (such as GPUs) to improve the training and testing speed,and shorten training and testing time.Secondly,the deep neural network model usually contains a large number of parameters that require high-capacity,high-speed memory to store.These limitations hinder the widespread use of deep neural networks.At present,training and testing of deep neural networks usually run under high-performance servers or clusters.In some mobile devices with high real-time requirements,such as mobile phones,applications are limited.This paper reviewed the progress of compression deep neural network algorithm in recent years,and introduced the main me- thods of compressing neural network,such as cropping method,sparse regularization method,decomposition method,shared parameter method,mask acceleration method and discrete cosine transform method.Finally,the future research direction of compressed deep neural network was prospected.
作者
李青华
李翠平
张静
陈红
王绍卿
LI Qing-hua;LI Cui-ping;ZHANG Jing;CHEN Hong;WANGShao-qing(Key Laboratory of Data Engineering and Knowledge Engineering (Renmin University of China),Ministry of Education,Beijing 100872,China;School of Information,Renmin University of China,Beijing 100872,China;School of Computer Science and Technology,Shandong University of Technology,Zibo,Shandong 255091,China)
出处
《计算机科学》
CSCD
北大核心
2019年第9期1-14,共14页
Computer Science
关键词
深度学习
神经网络
模型压缩
Deep learning
Neural network
Model compression