真实世界超分辨率—语义分割联合框架研究

Study of the Joint Framework for Real-World Super-Resolution-Semantic Segmentation

现有的语义分割方法在干净的图像上可以产生较好的结果,但是在干净图像上训练的分割模型应用到真实世界的图像上时则会出现性能下降,这是因为训练域和测试域之间存在域间隙,从而降低了分割的准确性。针对真实世界语义分割的问题,本文提出了一种超分辨率—语义分割联合框架,用于提升语义分割准确性。具体来说,所提出的框架嵌入了一个两分支网络,其中包括超分辨率分支、语义分割分支和一个特征共享模块。超分辨率任务鼓励网络找到对不同分辨率特征鲁棒的表示,从而分割头部可以使用恢复的“干净”特征进行更好的预测。其中超分辨率分支仅配置在训练过程中,在推理阶段可以丢弃。基于构建的伪真实配对数据集CityDeg进行监督训练,提出的框架联合现有先进的语义分割方法能够在不引入额外计算成本的情况下有效提高低分辨率场景语义分割性能。

Existing semantic segmentation methods produce better results on clean images,but segmentation models trained on clean images applied to real-world images experience performance degradation because of the domain gap between the training and testing domains,which reduces the segmentation accuracy.To address the problem of real-world semantic segmentation,this paper proposes a joint super-resolution-semantic segmentation framework for improving semantic segmentation accuracy.Specifically,the proposed framework embeds a two-branch network that includes a super-resolution branch,a semantic segmentation branch,and a feature sharing module.The super-resolution task encourages the network to find a robust representation of features with different resolutions,so that the segmentation head can use the recovered“clean”features for better prediction.The super-resolution branch is configured only during training and can be discarded during the inference phase.Based on the constructed pseudo-real pairwise dataset CityDeg for supervised training,the proposed framework,together with the existing state-of-the-art semantic segmentation methods,is able to effectively improve the performance of semantic segmentation for low-resolution scenes without introducing additional computational cost.