基于高度有效驱动注意力与多层级特征融合的城市街景语义分割

Urban street view semantic segmentation based on height-driven effective attention and multi-stage feature fusion

针对DeepLabv3+网络在进行城市街景图像分割任务时,没有充分利用到网络中多层级特征信息,导致分割结果存在大目标有孔洞、边缘目标分割不够精细等不足;并且考虑到城市街景数据具有天然的空间位置特殊性,本文提出在DeepLabv3+网络的基础上引入高度有效驱动注意力机制(height-driven efficient attention model,HEAM)与多层级特征融合模块(multi-stage feature fusion model,MFFM),将HEAM嵌入特征提取网络与空洞空间金字塔池化(atrous spatial pyramid pooling,ASPP)结构中,使其对目标关注更多垂直方向上的空间位置信息;MFFM通过融合多层特征图,在网络中形成多条融合支路依次连接到网络解码端,采用逐次上采样提高解码时像素上的连续性。将改进的网络通过CamVid城市街景数据集验证测试,实验结果表明,该网络能有效改善DeepLabv3+的不足,并且合理运用了数据集的位置先验性,增强了分割效果,在CamVid测试集上平均交并比(mean intersection over union,MIoU)达到了68.2%。

Deeplabv3+network does not make full use of multi-stage feature information in urban street view image segmentation,which leads to the shortcomings of large targets with holes,imprecise segmentation of edge target and so on.Considering the natural spatial position particularity of urban street view data,this paper proposes to introduce a height-driven effective attention model(HEAM)and a multi-stage feature fusion model(MFFM)on the basis of Deeplabv3+network,and it embeds HEAM into the feature extraction network and atrous spatial pyramid pooling(ASPP)structure,which makes it pay attention to more spatial position information in the vertical direction.MFFM integrates multi-layer feature images to form multiple branches in the network and connect them to the network decoding end in turn.Successive up-sampling is used to improve the continuity of pixels during decoding.The improved network is verified and tested by CamVid urban street view data set.The results show that the network can effectively improve the deficiency of DeepLabv3+,and the location priori of the data set is properly used to enhance the segmentation effect.Mean intersection over union(MIoU)on CamVid test set reaches 68.2%.