基于多任务学习的全景分割方法研究

Research on Panoptic Segmentation Method Based on Multi-task Learning

本文提出了一种用于全景分割的多任务学习深层神经网络,其目标是为输入图像的每个像素提供类别标签和实例编号.该网络模型同时进行语义分割和实例分割预测,并把他们的预测结果组合起来形成全景分割的输出.首先,使用带有特征金字塔网络的ResNet50主干网络进行特征提取;然后,把提取的特征分别用于语义分割及实例分割分支,并在两个任务之间共享信息;最后,通过空间排序模块把两个子任务的输出结果融合,得到全景分割的最终输出结果.本文的网络模型在数据集Cityscapes与COCO上进行了训练与测试,实验结果表明,通过使用多任务学习方法,可以有效提高整个模型预测结果的准确率.

This paper proposes a multi-task learning deep neural network for panoptic segmentation,the goal of which is to provide a category label and instance number for each pixel of the input image.The network model carry out the prediction of semantic segmentation and instance segmentation at the same time.,and the predictions are combined as the panoptic segmentation's output.We performed feature extraction by using the ResNet50 backbone network with a feature pyramid network.The extracted features are used for semantic segmentation and instance segmentation,while sharing informations between the two tasks.We merged the output results of the two subtasks through the spatial sorting module to obtain the final output result of the panoptic segmentation.This paper trains and tests the network model on the Cityscapes and COCO.The experimental results show that the accuracy of the prediction results of the entire model can be improved by using multi-task learning method.