ConvFormer:基于Transformer的视觉主干网络

ConvFormer:Vision Backbone Network Based on Transformer

针对主流Transformer网络仅对输入像素块做自注意力计算而忽略了不同像素块间的信息交互,以及输入尺度单一导致局部特征细节模糊的问题,本文提出一种基于Transformer并用于处理视觉任务的主干网络ConvFormer. ConvFormer通过所设计的多尺度混洗自注意力模块(Channel-Shuffle and Multi-Scale attention,CSMS)和动态相对位置编码模块(Dynamic Relative Position Coding,DRPC)来聚合多尺度像素块间的语义信息,并在前馈网络中引入深度卷积提高网络的局部建模能力.在公开数据集ImageNet-1K,COCO 2017和ADE20K上分别进行图像分类、目标检测和语义分割实验,ConvFormer-Tiny与不同视觉任务中同量级最优网络RetNetY-4G,Swin-Tiny和ResNet50对比,精度分别提高0.3%,1.4%和0.5%.

To solve the problem that the mainstream network based on Transformer only does self-attention com-putation on the input pixel blocks and ignores the information interaction between different pixel blocks,as well as the blurring of local feature details due to a single input scale,a backbone network based on Transformer and used for pro-cessing vision tasks is proposed called ConvFormer.ConvFormer aggregates the semantic information between multi-scale pixel blocks through the designed channel-shuffle and multi-scale attention(CSMS)and dynamic relative position coding(DRPC)modules,as well as introduces deep convolution in the feedforward network to improve the local model-ing capability of the network.In the image classification,target detection,and semantic segmentation experiments on public datasets ImageNet-1K,COCO 2017,and ADE20K,ConvFormer-Tiny compares with the optimal networks of the same magnitude RetNetY-4G,Swin-Tiny,and ResNet50 in different vision tasks,the accuracy is improved by 0.3%,1.4%,and 0.5%.