深度神经网络的仿生矩阵约简与量化方法

Bio-inspired matrix reduction and quantization method for deep neural network

基于生物学原理的深度神经网络(DNN)的发展给人工智能领域带来了革命性的突破,然而当前神经网络的发展却越来越脱离生物学原理,DNN越来越臃肿的模型对存储空间和计算力的需求越来越高,并且对于DNN在嵌入式/移动端设备上的部署带来了阻碍。针对这一问题,对生物学进化选择原理进行研究,并提出一种基于"进化"+"随机"+"选择"的全新神经网络算法。该方法在保持现有神经网络模型的基本框架的前提下,能极大简化现有模型的大小。首先对权值参数进行聚类,然后在参数的聚类质心值的基础上添加随机微扰进行参数重构,最后通过对重构模型进行图像分类和目标检测来实现准确度测试以及模型稳定性分析。在ImageNet数据集和COCO数据集上的实验结果表明,提出的模型重构方法在对图像分类和目标检测的测试准确度提升1%~3%的情况下,仍可将Darknet19、ResNet18、ResNet50以及YOLOv3等四种重构模型的体量压缩到原来的1/4~1/3,并还有进一步简化的可能。

Bio-inspired Deep Neural Network(DNN)is a revolutionary breakthrough in artificial intelligent field.However,the lack of storage space as well as computing capacity caused by the explosive increase of the model weights not only keeps DNN apart from its original inspiration,but also makes it difficult to deploy DNN on embedded/mobile devices.In order to solve this problem,the biological selection principle in the evolution was studied,and a novel neural network algorithm based on“evolution”+“randomness”+“selection”was proposed.In this method,the size of the existing models were greatly simplified on the premise of maintaining the basic framework of the existing neural network models.First,the weight parameters were clustered.Then,based on the cluster centroid values of the parameters,the random perturbation was added to reconstruct the parameters.Finally,the image classification and object detection were performed on the reconstructed model to realize the accuracy test and model stability analysis.Experimental results on ImageNet dataset and COCO dataset show that the proposed model reconstruction method can compress the sizes of four models,including Darknet19,ResNet18,ResNet50 and YOLOv3,to 1/4-1/3 of the original ones,and under the condition of 1%-3%performance improvement in the test accuracy of image classification and object detection,there is the possibility of further simplification.