深度神经网络参数轻量化方法综述

An Overview on Lightweight Methods for Deep Neural Network Parameters

近年来,深度神经网络在各种具有挑战性的任务上取得了巨大的成功,不断刷新人们对人工智能的认识。但是,深度神经网络模型的参数量巨大、计算成本、存储成本过高,难以部署到资源受限的边缘计算设备中。因此,人们开始从网络的架构和参数量两个角度尝试对网络进行轻量化设计,同时保证神经网络性能可接受。本文从网络参数轻量化的角度出发,首先简要回顾深度神经网络发展历史和工作原理;其次,介绍当前主流的3类参数轻量化方法:参数量化、张量分解以及参数共享;然后,从思想描述、适用层级、训练方式等维度对比分析方法优劣;最后,对神经网络轻量化的未来发展方向进行展望。

In recent years,deep neural networks have achieved great success on a variety of challenging tasks,constantly refreshing people’s understanding of artificial intelligence.However,the deep neural network model is difficult to deploy to resource-limited edge computing devices because of its huge parameters,high computing and storage costs.Therefore,people begin to try to lightweight network design from the perspective of network architecture and parameter number,while ensuring the performance of neural network is acceptable.In this paper,the development history and working principle of deep neural networks are reviewed briefly from the angle of network parameter lightweight.Secondly,three kinds of current mainstream parameter lightweight methods are introduced:parameter quantization,tensor decomposition and parameter sharing.Then the advantages and disadvantages of the method are compared and analyzed from the dimensions of thought description,application level and training mode.Finally,the future development direction of neural network lightweight is prospected.