Real-time instance segmentation based on contour learning

基于轮廓学习的实时实例分割

Instance segmentation plays an important role in image processing.The Deep Snake algorithm based on contour iteration deforms an initial bounding box to an instance contour end-to-end,which can improve the performance of instance segmentation,but has defects such as slow segmentation speed and sub-optimal initial contour.To solve these problems,a real-time instance segmentation algorithm based on contour learning was proposed.Firstly,ShuffleNet V2 was used as backbone network,and the receptive field of the model was expanded by using a 5×5 convolution kernel.Secondly,a lightweight up-sampling module,multi-stage aggregation(MSA),performs residual fusion of multi-layer features,which not only improves segmentation speed,but also extracts effective features more comprehensively.Thirdly,a contour initialization method for network learning was designed,and a global contour feature aggregation mechanism was used to return a coarse contour,which solves the problem of excessive error between manually initialized contour and real contour.Finally,the Snake deformation module was used to iteratively optimize the coarse contour to obtain the final instance contour.The experimental results showed that the proposed method improved the instance segmentation accuracy on semantic boundaries dataset(SBD),Cityscapes and Kins datasets,and the average precision reached 55.8 on the SBD;Compared with Deep Snake,the model parameters were reduced by 87.2%,calculation amount was reduced by 78.3%,and segmentation speed reached 39.8 frame·s−1 when instance segmentation was performed on an image with a size of 512×512 pixels on a 2080Ti GPU.The proposed method can reduce resource consumption,realize instance segmentation tasks quickly and accurately,and therefore is more suitable for embedded platforms with limited resources.

实例分割在图像处理中发挥着重要作用,基于轮廓迭代的Deep Snake算法实现初始目标框到目标轮廓端到端变形,能提升实例分割的性能,但存在分割速度较慢、初始轮廓欠优等缺陷。针对这些问题,提出了一种轻量化的实例轮廓迭代分割算法。首先,采用ShuffleNet V2作为主干网络,使用5×5卷积核扩大了模型感受野。其次,设计了轻量级上采样模块—多阶段聚合(Multi stage aggregation,MSA)进行多层特征的残差融合,不仅提高了分割速度,而且更加全面地提取了有效特征。再次,设计了一种网络学习的轮廓初始化方法,采用全局轮廓特征聚合机制回归得到粗糙轮廓,解决了手工初始化轮廓和真实轮廓误差过大的问题。最后,使用Snake模型对粗糙轮廓进行迭代优化得到最终的实例轮廓。实验结果表明,所提出的方法在Semantic boundaries dataset(SBD)、Cityscapes和Kins数据集上提高了实例分割精度,在SBD数据集上平均分割精度达到55.8,同时模型参数量较Deep Snake减少了87.2%,计算量减少了78.3%,在2080Ti GPU上对512×512像素的图片进行实例分割时速度达到39.8帧·s−1。该方法能够减少资源消耗,快速准确实现实例分割任务,更适合应用于资源有限的嵌入式平台。