基于Mask R-CNN和多特征融合的实例分割

Instance Segmentation Modal Based on Mask R-CNN and Multi-feature Fusion

为了能够充分地利用图像特征信息,提升实例分割的效果,提出了一种基于Mask R-CNN网络结构和多特征融合的实例分割模型。首先,在Mask R-CNN模型的基础上引入两条分支:一条基于整体嵌套边缘检测(HED)模型的边缘检测分支生成偏重于边缘信息的边缘特征图,一条基于全卷积网络(FCN)的语义分割分支生成偏重于空间位置信息的语义特征图。然后,在进行感兴趣区域对齐(ROIAlign)时,为了充分利用特征金字塔的各层信息,将感兴趣区域(ROI)同时映射到相应的金字塔层及其相邻层。最后,融合以上得到的多个特征图,生成信息更加丰富的新特征用于后续的检测和分割任务。实验结果表明,该方法有效提高了检测和分割的准确性。在使用Resnet50-FPN作为骨干网络且没有附加条件的情况下,与Mask R-CNN相比,该模型的检测和分割平均精度(mAP)分别提升了1.2%和1.0%。

To fully utilize image features to improve the effect of instance segmentation,an instance segmentation model based on Mask R-CNN network structure and multi-feature fusion scheme is proposed.Firstly,two branches are introduced on the basis of Mask R-CNN.One is an edge detection branch based on holistically-nested edge detection(HED)model to generate edge feature graph with emphasis on edge information,the other is a semantic segmentation branch based on fully convolution network(FCN)to generate semantic feature graph with emphasis on rich spatial location information.Secondly,when performing ROIAlign,regions of interest(ROI)are mapped to the corresponding pyramid layer and its adjacent layers to make full use of the information of each layer of the feature pyramid.Finally,the above multiple feature graphs are fused,and the new features with richer information can be generated for subsequent detection and segmentation tasks.Experiment shows that the proposed method effectively improves the accuracy of detection and segmentation.With Resnet50-FPN as the backbone network and no bells and whistles,the box AP is increased by 1.2%and the mask AP is increased by 1.0%compared to Mask R-CNN.