基于自监督图像对的弱监督语义分割算法

Weakly supervised semantic segmentation algorithm based on self-supervised image pair

为了降低语义分割任务的标注成本,提出一种基于自监督图像对的弱监督语义分割算法Co-Net。首先,将一对图像分别输入骨干网络中提取图像对特征;然后,将特征展开加入位置信息送入编码层中进行编码;接着,将编码特征送入协同注意力模块(CoAM)以及双向自注意力模块(BiAM)中进行信息相互表征;最后,将图像区域掩码模型(MRM)以及图像对匹配(IPM)两种自监督任务用于网络训练,学习图像对中的全局关联以及局部关联,以此得到更加精确的初始化种子。仅使用图像级标签进行弱监督语义分割,在Pascal VOC 2012验证和测试集上分别实现了69.8%和70.3%的平均交并比(mIoU),相较于同样为图像对输入的算法GroupWSSS(Group-Wise Semantic mining for weakly Supervised Semantic Segmentation),验证集、测试集上的mIoU分别提高了1.6、1.8个百分点。实验结果表明,所提算法可以获得更加完整的目标激活区域。

In order to reduce annotation cost in semantic segmentation tasks,a new weakly supervised semantic segmentation algorithm Co-Net(Collaborative Network)based on self-supervised image pairs was proposed.Fistly,a pair of images were respectively input into backbone network to extract image pair features.Secondly,expanded features were added to location information and sent to the encoding layer.Then,the encoded features were fed into Collaborative Attention Module(CoAM)and Bi-directional self-Attention Module(BiAM)for information mutual representation.Finally,two self-supervised tasks,image Mask Region Model(MRM)and Image Pair Matching(IPM)were used for network training to learn global and local associations in image pairs,so as to obtain more accurate initialization seeds.Weakly supervised semantic segmentation using only image-level labels achieved mean Intersection over Union(mIoU)of 69.8%and 70.3%on Pascal VOC 2012 validation and test sets.respectively.Compared with the algorithm Group-Wise Semantic mining for weakly Supervised Semantic Segmentation(GroupWSSS),which is also input for image pairs,mIoU was increased by 1.6 and 1.8 percentage points on the validation set and test set.The experimental results show that the proposed algorithm can obtain a more complete target activation area.