UG_2F2F:多特征深度融合的人脸图像修复

UG_2F2F:Face Image Inpainting Based on Deep Fusion of Multiple Features

利用现有深度学习方法实现人脸图像修复存在特征挖掘不充分、语义表达不完整等问题,导致输出图像容易存在伪影或模糊纹理等现象.为了解决这个问题,提出一种多特征深度融合的人脸图像修复模型.以融入门控卷积的U-Net作为主干网络,来提取结构和纹理这两种特征.再对两特征依次使用双向门控特征融合和门控注意特征融合进行两次融合,以充分挖掘图像高级语义及特征间上下文关系,实现精准而且有效的空洞填充.在训练过程中,定义一种修正的重建损失函数,强调保持结构完整时生成更多纹理细节.在CelebA-HQ数据集上的实验结果表明,与CA、EdgeConnect和CTSDG等代表性模型相比,所提出的图像修复模型在峰值信噪比、结构相似度和FID指标上均得到提升,它能够有效修复人脸图像.

Using existing deep learning methods to fill holes generally result in the problems such as feature insufficient mining and incomplete semantic expression.Therefore,the output image is prone to artifacts or blurred textures.To solve mainly these problems,a face image inpainting model with multi-feature deep fusion was proposed.The U-Net incorporating gated convolution is used as the backbone network to extract both structure and texture features.Then,the two features are fused twice using Bi-directional Gated Feature Fusion and Gated Attentional Feature Fusion to achieve the hole filling.This filling process is precise and effective.It's benefit to fully mining the high-level semantics of image and contextual relationships between features.During the training process,a modified reconstruction loss function is defined,which emphasizes generating more texture details while keeping the complete structure.Experiments on CelebA-HQ dataset demonstrate that the proposed model's peformances such as peak signal-to-noise ratio,structural similarity index and FID indicator are promoted,relative to these representative models include CA,EdgeConnect and CTSDG etc.Proposed model could inpaint face image effectively.