基于语义先验和双通道特征提取的图像修复

Image restoration based on semantic prior and dual channel feature extraction

针对现有的图像修复算法重建结果存在的局部结构不连通、细节还原不准确等问题,提出了一种基于语义先验和双通道特征提取的图像修复算法(semantic prior and dual channel extraction, SPDCE)。该算法利用语义先验网络学习缺失区域的语义信息和上下文知识,对缺失区域进行预测,增强了生成图像的局部一致性;然后通过双通道特征提取网络充分挖掘图像信息,提升了对纹理细节的感知和利用能力;再使用上下文特征调整模块在多个尺度上捕获并编码丰富的语义特征,从而生成更真实的图像视图和更精细的纹理细节。在CelebA-HQ和Places2数据集上的实验结果表明,SPDCE算法与常用算法相比,峰值信噪比(peak signal-to-noise ratio, PSNR)和结构相似性(structural similarity, SSIM)分别提升1.6~1.73 dB和3.1%~9.9%,L_1 loss下降15.2%~27.8%。实验证明所提算法修复后的图像具有更合理的结构和更丰富的细节,图像修复效果更优。

Aiming at the problems of local structure disconnection and inaccurate detail restoration in existing image restoration algorithms,this paper proposed an inpainting algorithm based on semantic prior and dual channel feature extraction(SPDCE).The algorithm used semantic prior network to learn the semantic information and context knowledge of the missing area,predicted the missing area,and enhanced the local consistency of the generated image.Then,the dual channel feature extraction network fully mined image information,enhanced the perception and utilization of texture details.Next,the context feature adjustment module could capture and encode rich semantic features at multiple scales,thereby generating more realistic image views and finer texture details.After conducting experimental verification on the datasets CelebA-HQ and Places2,the results show that compared with commonly used algorithms,the SPDCE algorithm improved the peak signal ratio(PSNR)and structural similarity(SSIM)by 1.6 dB to 1.73 dB and 3.1%to 9.9%,while L 1 loss decreased by 15.2%to 27.8%.Experimental results show that the repaired image of the proposed algorithm has more reasonable structure and richer details,and the inpainting effect is better.