Winograd快速卷积相关研究综述

Review of Winograd Fast Convolution Technique Research

卷积神经网络(CNN)已经被广泛应用到各个领域并发挥了重要作用。卷积算子是卷积神经网络的基础组件,同时也是最耗时的部分。近年来,研究者提出了包括基于FFT和Winograd的若干种快速卷积算法。其中Winograd卷积因大幅减少了卷积中乘法操作且占用内存更小而迅速成为小卷积核的卷积算子上快速卷积实现的首选。但目前相关工作聚焦于算法的一般化、拓展和各类体系结构上的实现,还没有研究者对Winograd卷积算法作系统性的总结。为了给后续研究者提供详细的参考依据,对Winograd卷积引入以来的相关工作进行了总结。首先阐述了Winograd最小滤波算法及Winograd卷积的引入,介绍了Winograd卷积的一般化与拓展,并对比了现有实现之间的差异;从稀疏剪枝、低精度与量化、数值稳定性这三方面介绍了Winograd卷积的优化工作,并详细介绍了相关具体方法的优缺点;对各类体系结构上的实现和优化进行了分类总结,比较了各平台上实现可用的通用优化方法,并介绍了Winograd卷积的实际应用;最后对内容进行了简要总结,分析了现有研究的局限性,并对未来可能的方向进行了初步展望。

Convolutional neural networks(CNN)have been widely used in various fields and have played an impor tant role.Convolution operator is the basic component of CNN,and it is also the most time-consuming part.In recent years,researchers have proposed several fast convolution algorithms including FFT and Winograd.Among them,Winograd convolution has quickly become the first choice for fast convolution implementation on convolu tion operators with small convolution kernels,because it greatly reduces the multiplication operations in convolu tion and occupies less memory.Related work focuses on the generalization,extension and implementation on various architectures of the Winograd convolution,but there are no researchers who have systematically summa rized the Winograd convolution algorithm.This paper aims to provide detailed reference for follow-up researchers,and summarizes all related work since the introduction of Winograd convolution.Firstly,the introduction of Winograd minimum filtering algorithm and Winograd convolution is described,the generalization and extension of Winograd convolution are introduced,and the detailed differences between existing implementations are also listed.The optimization of Winograd convolution is introduced from the three aspects of sparse pruning,low precision and quantization,and numerical stability,and the advantages and disadvantages of the specific methods are elaborated.The implementations and optimizations of various architectures are classified and summarized,the general optimi zation methods available for implementation on each platform are compared,and the practical application of Winograd convolution is also introduced.Finally,a brief summary of the content is made,the limitations of existing research are analyzed,and a preliminary outlook for the possible future directions is made.