基于张量虚拟机的快速卷积自动性能优化

Fast Convolution Automatic Performance Optimization Based on Tensor Virtual Machine

卷积神经网络作为深度学习的典型代表,是计算机视觉等任务中最常用的神经网络,然而,卷积运算通常占整个卷积神经网络运行时的90%以上,成为卷积神经网络的性能瓶颈。此外,由于当下硬件的复杂性及工作负载的多样性,之前工作中的一些特定优化往往缺乏性能可移植性。对此,作者提出BlazerML,一个基于张量虚拟机(TVM)模板代码自动生成的开源卷积计算库,可为任何输入形状自动生成高性能的卷积实现。BlazerML是基于Winograd算法实现的,因为该算法是快速卷积算法中性能最高的算法。实验结果表明:BlazerML显著优于当下最先进的开源库。在x86 CPU上运行常见的深度学习网络前向推理分别比OnnxRuntime、MNN和TVM社区版本快1.18~2.47倍、1.18~2.27倍和1.01~1.66倍。在ARMCPU上运行常见深度学习网络的单层推理分别比ACL和FastConv快1.26~6.11倍、1.04~4.28倍。

Convolutional Neural Networks(CNNs)as a quintessential representation of deep learning,are the most commonly used neural networks in tasks such as computer vision.However,convolution operations typically account for over 90%of the runtime in CNNs,becoming a bottleneck for performance.Additionally,due to the complexity of current hardware and the diversity of workloads,specific optimizations in previous work often lack performance portability.To address this problem,the author introduces BlazerML,an open-source convolution computation library based on auto-generated code templates from TVM,capable of automatically generating high-performance convolution implementations for any input shape.BlazerML is implemented based on the Winograd algorithm,known for its high performance in fast convolution algorithms.Experimental results demonstrate that BlazerML significantly outperforms current state-of-the-art open-source libraries.On x86 CPUs,running common deep learning network forward inferences,it is faster by 1.18—2.47 times,1.18—2.27 times,and 1.01—1.66 times compared to OnnxRuntime,MNN,and the TVM community version,respectively.On ARM CPUs,for single-layer inference of common deep learning networks,it surpasses ACL and FastConv by 1.26—6.11 times and 1.04—4.28 times,respectively.