注意力引导多模态融合的RGB-D图像分割

Attention-guided multi-modal fusion RGB-D image segmentation

为提高图像分割效率,提出注意力引导多模态交叉融合分割网络(ACFNet)。采用编码器-解码器结构,设计非对称双流特征提取网络,RGB和深度编码器分别以ResNet-101和ResNet-50为主干网络,并在RGB编码器中添加全局-局部特征提取模块(GL)。为有效融合RGB和深度特征,提出注意力引导多模态交叉融合模块(ACFM),在多阶段利用融合的增强特征表示。实验结果表明,ACFNet在室内场景分割数据集NYUD V2上的平均交并比(mIou)达到了51.5%,与先进的语义分割算法相比,显著提高了分割性能。

To improve the efficiency of image segmentation,an attention-guided multi-modal cross fusion segmentation network(ACFNet)was proposed.An encoder-decoder structure was used.An asymmetric dual-stream feature extraction network was designed with RGB and depth encoders using ResNet-101 and ResNet-50 respectively as the backbone network,and a global-local feature extraction module(GL)was added to the RGB encoder.To effectively fuse RGB and depth features,an attention-guided multi-modal cross fusion module(ACFM)was proposed to better utilize the fused enhanced feature representation in multiple stages.Experimental results show that the mean intersection-over-union(mIou)of ACFNet on the indoor scene segmentation dataset NYUD V2 reaches 51.5%,which significantly improves the segmentation effect compared with advanced semantic segmentation algorithms.