基于张量虚拟机的深度神经网络模型加速方法

Deep neural network model acceleration method based on tensor virtual machine

随着人工智能(AI)技术的蓬勃发展,深度神经网络(DNN)模型被大规模应用到各类移动端与边缘端。然而,边缘端算力低、内存容量小,且实现模型加速需要深入掌握边缘端硬件知识,这增加了模型的部署难度,也限制了模型的推广应用。因此,基于张量虚拟机(TVM)提出一种DNN加速与部署方法,从而实现卷积神经网络(CNN)模型在现场可编程门阵列(FPGA)上的加速,并在分心驾驶分类应用场景下验证了所提方法的可行性。通过计算图优化方法减小了模型的访存和计算开销,通过模型量化方法减小了模型尺寸,通过计算图打包方法将卷积计算卸载到FPGA上执行以提高模型推理速度。与微处理器(MPU)相比,所提方法可使ResNet50和ResNet18在MPU+FPGA上的推理时间分别减少88.63%和77.53%;而在AUC(American University in Cairo)数据集上,相较于MPU,两个模型在MPU+FPGA上的top1推理精度仅下降了0.26和0.16个百分点。可见,所提方法可以降低不同模型在FPGA上的部署难度。

With the development of Artificial Intelligence(AI)technology,the Deep Neural Network(DNN)models have been applied to various mobile and edge devices widely.However,the model deployment becomes challenging and the popularization and application of the models are limited due to the facts that the computing power of edge devices is low,the memory capacity of edge devices is small,and the realization of model acceleration requires in-depth knowledge of edge device hardware.Therefore,a DNN acceleration and deployment method based on Tensor Virtual Machine(TVM)was presented to accelerate the Convolutional Neural Network(CNN)model on Field-Programmable Gate Array(FPGA),and the feasibility of this method was verified in the application scenarios of distracted driving classification.Specifically,in the proposed method,the computational graph optimization method was utilized to reduce the memory access and computational overhead of the model,the model quantization method was used to reduce the model size,and the computational graph packing method was adopted to offload the convolution calculation to the FPGA in order to speed up the model inference.Compared with MPU(MicroProcessor Unit),the proposed method can reduce the inference time of ResNet50 and ResNet18 on MPU+FPGA by 88.63%and 77.53%respectively.On AUC(American University in Cairo)dataset,compared to MPU,the top1 inference accuracies of the two models on MPU+FPGA are only reduced by 0.26 and 0.16 percentage points respectively.It can be seen that the proposed method can reduce the deployment difficulty of different models on FPGA.