基于改进BiSeNet的实时图像语义分割

Real-time semantic segmentation based on improved BiSeNet

为了提升图像语义分割算法的性能,使其同时满足准确性和实时性需求,本文提出了一种基于改进BiSeNet的实时图像语义分割算法。首先,通过使双分支网络头部共享以消除BiSeNet网络结构部分通道和参数的冗余,同时有效提取图像的浅层特征;然后,将上述共享网络拆分为由细节分支和语义分支组成的双分支网络,并分别用于提取空间细节信息和语义上下文信息;此外,在语义分支尾部引入通道和空间注意力机制以增强特征表达能力,通过使用双注意力机制对BiSeNet算法进行优化以更有效地提取语义上下文特征;最后,对细节分支和语义分支的特征进行融合并通过上采样操作恢复至输入图像分辨率大小以实现图像语义分割。本文算法在Cityscapes数据集以95.3FPS的实时性表现达到77.2%mIoU的准确性;在CamVid数据集以179.1 FPS的实时性表现达到73.8%mIoU的准确性。实验结果表明,本文算法在实时性和准确性方面获得了很好的平衡,其语义分割性能相较于BiSeNet算法及其它现有算法得到了显著的提升。

To improve the performance of image semantic segmentation on accuracy and efficiency for practical applications,in this study,we propose a real-time semantic segmentation algorithm based on improved BiSeNet.First,the redundancy of certain channels and parameters of BiSeNet is eliminated by sharing the heads of dual branches,and the affluent shallow features are effectively extracted at the same time.Subsequently,the shared layers are divided into dual branches,namely,the detail branch and the se⁃mantic branch,which are used to extract detailed spatial information and contextual semantic information,respectively.Furthermore,both the channel attention mechanism and spatial attention mechanism are in⁃troduced into the tail of the semantic branch to enhance the feature representation;thus the BiSeNet is opti⁃mized by using dual attention mechanisms to extract contextual semantic features more effectively.Final⁃ly,the features of the detail branch and semantic branch are fused and up-sampled to the resolution of the input image to obtain semantic segmentation.Our proposed algorithm achieves 77.2%mIoU on accuracy with real-time performance of 95.3 FPS on Cityscapes dataset and 73.8%mIoU on accuracy with realtime performance of 179.1 FPS on CamVid dataset.The experiments demonstrate that our proposed se⁃mantic segmentation algorithm achieves a good trade-off between accuracy and efficiency.Furthermore,the performance of semantic segmentation is significantly improved compared with BiSeNet and other ex⁃isting algorithms.