紧致化神经网络的鲁棒性分析

Robustness analysis for compact neural networks

深度神经网络(deep neural network,DNN)目前已经在很多视觉识别任务中达到了超越人类性能的表现.然而,将深度神经网络部署到真实场景的终端设备面临着两个挑战:首先,目前的深度神经网络模型非常耗费计算资源和内存,很难直接应用在资源受限的终端设备上;其次,真实场景的数据往往受到各种各样噪声的影响,因此部署在真实场景的模型应该拥有良好的鲁棒性(robustness).近年来出现了很多对深度神经网络进行压缩和加速的方法以适应真实场景的资源受限终端,而对这些压缩模型的鲁棒性分析也得到了越来越多的关注.本文首次对神经网络剪枝、量化及知识蒸馏等常用的压缩算法进行对抗鲁棒性及噪声鲁棒性分析,总结了不同压缩算法对神经网络鲁棒性的影响.并对目前同时解决神经网络鲁棒性及模型大小问题的一些方案进行了整理与分析,展望了该领域未来研究的技术挑战与可能的发展方向.

Deep neural networks(DNNs)have achieved comparable performance to humans on many tasks.However,there are two problems in deploying a DDN to terminal devices of a real scene.First,DNNs consume huge computing resources and memory;therefore,it is difficult to directly apply DNNs to resource-constrained terminal devices.Second,real-world data are often affected by noise;accordingly,the model deployed in the real scene needs to have good robustness.Recently,many compression methods for DNNs have been adapted to resource-constrained terminal devices,and robustness analyses of these compression models have increasingly received attention.In this paper,the adversarial and corruption robustness of several compression algorithms,such as pruning,quantization,and knowledge distillation,is first analyzed.Then,several studies of compact networks with improved robustness are summarized.Finally,several challenges are discussed,and possible research directions for compact robustness networks are proposed.