NN-EdgeBuilder:面向边缘端设备的高性能神经网络推理框架

NN-EdgeBuilder:High-performance Neural Network Inference Framework for Edge Devices

飞速发展的神经网络已经在目标检测等领域取得了巨大的成功,通过神经网络推理框架将网络模型高效地自动部署在各类边缘端设备上是目前重要的研究方向。针对以上问题,该文设计一个针对边缘端FPGA的神经网络推理框架NN-EdgeBuilder,能够利用基于多目标贝叶斯优化的设计空间探索算法充分探索网络每层的并行度因子和量化位宽,接着调用高性能且通用的硬件加速算子来生成低延迟、低功耗的神经网络加速器。该文使用NN-EdgeBuilder在Ultra96-V2 FPGA上部署了UltraNet和VGG网络,生成的UltraNet-P1加速器与最先进的Ul-traNet定制加速器相比,功耗和能效比表现分别提升了17.71%和21.54%。与主流的推理框架相比,NN-Edge-Builder生成的VGG加速器能效比提升了4.40倍,数字信号处理器(DSP)的计算效率提升了50.65%。

The rapidly developing neural network has achieved great success in fields such as target detection.Currently,an important research direction is to deploy efficiently and automatically network models on various edge devices through a neural network inference framework.In response to these issues,a neural network inference framework NN-EdgeBuilder for edge FPGA is designed in this paper,which can fully explore the parallelism factors and quantization bit widths of each layer of the network through a design space exploration algorithm based on multi-objective Bayesian optimization.Then high-performance and universal hardware acceleration operators are called to generate low-latency and low-power neural network accelerators.NN-EdgeBuilder is used to deploy UltraNet and VGG networks on Ultra96-V2 FPGA in this study,and the generated UltraNet-P1 accelerator improves power consumption and energy efficiency by 17.71%and 21.54%,respectively,compared with the state-of-the-art UltraNet custom accelerator.Compared with mainstream inference frameworks,energy efficiency of the VGG accelerator generated by NN-EdgeBuilder is improved by 4.40 times and Digital Signal Processor(DSP)computing efficiency is improved by 50.65%.