基于双分支交互的实时语义分割算法

Real-time semantic segmentation algorithm based on two-branch interaction

针对双分支实时语义分割算法存在双分支交互差、多尺度上下文信息提取不完善等问题,提出了基于双分支交互的实时语义分割网络(dual-branch interactive multi-scale fusion network for real-time semantic segmentation,DIMFNet)。算法以引导聚合双边语义分割网络(bilateral network with guided aggregation for real-time semantic segmentation,BiseNetV2)的双分支结构为基准进行改进,空间分支提取空间细节特征,上下文分支提取深层上下文特征。结合注意力思想提出注意力引导高级语义融合模块(attention guide high-level semantics fusion module,AGHSM)实现双分支的交互融合,以获得更好的空间特征表示;对金字塔池化模块进行改进,提出采用多层聚合金字塔池化模块(multi-layer aggregation pyramid pooling module,MAPPM)提取多尺度上下文特征,以获得更好的上下文特征表示。算法在Cityscapes数据集上进行消融实验并与现有实时语义分割网络进行对比,验证了各模块的有效性,以124.5 f/s达到了77.9%的平均交并比(mean intersection over union,MIoU);在CamVid数据集上以211.1 f/s达到了75.1%的MIoU。相比现有的实时语义分割网络,本文算法更好地权衡了分割的精度和速度。

In response to current issues of the dual-branch real-time semantic segmentation algorithm,such as poor interaction between the two branches and incomplete extraction of multi-scale contextual information,this paper proposes the dual-branch interactive multi-scale fusion network for real-time semantic segmentation(DIMFNet).The algorithm is based on the dual-branch structure of the bilateral network with guided aggregation for real-time semantic segmentation(BiseNetV2),with the spatial branch extracting spatial detail features and the context branch extracting deep contextual features.An attention guided high-level semantics fusion module(AGHSM)is proposed to achieve interactive fusion of the dual branches using the attention mechanism,so as to obtain better spatial feature representation.Furthermore,the algorithm improves the pyramid pooling module and introduces the multi-layer aggregation pyramid pooling module(MAPPM)to extract multi-scale contextual features,obtaining better contextual feature representation.The algorithm conducts ablation experiments on the Cityscapes dataset and is compared with existing real-time semantic segmentation networks,verifying effectiveness of each module.It achieves an average intersection over union(MIoU)of 77.9%at a speed of 124.5 frames per second(f/s)on the Cityscapes dataset,and 75.1%MIoU at 211.1 f/s on the CamVid dataset.Compared with existing real-time semantic segmentation networks,the proposed algorithm can better balance segmentation accuracy and speed.