基于混合空洞卷积网络的多鉴别器图像修复

Multi-discriminator image inpainting algorithm based on hybrid dilated convolution network

为了有效解决大面积语义信息缺失、孔洞区域大小及形状不规则、图像背景复杂时修复结果出现边缘模糊、伪影或修复失真等缺陷,提出了一种基于混合空洞卷积网络的多鉴别器图像修复算法.首先,将待修复图像输入一个基于混合空洞卷积层的模糊卷积网络,以重构损失为标准,进行粗修复.然后,将粗修复结果输入双平行卷积网络,该网络包含混合空洞卷积(HDC)层的卷积路径及一个与之平行的感知层卷积路径,两个平行路径的输出经过解码和反卷积后,送入鉴别器进行判别优化.最后,在网络的优化过程中,利用全局鉴别器、局部鉴别器和中心鉴别器增强修复图像的整体及局部语义一致性和细节特征.在国际公认的人脸数据集CelebA和风景数据集Places2上,对提出算法进行训练和测试,实验结果表明:提出方法在修复背景复杂和各种大小及形状的孔洞时,增强了图像细节的修复精度,有效避免了修复失真,在修复的视觉效果、峰值信噪比、结构相似度和平均误差方面,优于对比的4种经典修复算法.

In order to effectively solve the defects of large-area semantic information loss,the appearance of blurred edges and artifact or distortion when the lost area size is large and the shape is irregular under complex image background,a multiple discriminator based on hybrid dilated convolution network image inpainting algorithm was proposed.First,the image to be restored was input into a fuzzy convolution network based on hybrid dilated convolution layer,which the rough restoration was carried out on account of reconstruction loss. Subsequently,the rough restoration was fed into the bi-parallel convolution network,which contained the convolution path of HDC layer and a convolution path of the parallel contextual attention layer.After decoding and de-convolution,the outputs of the two parallel paths were sent to the discriminator for discrimination and optimization. Finally,in the process of network optimization,the global discriminator,local discriminator and central discriminator were utilized to enhance the overall and local semantic consistency and the detail features of the inpainted image.The proposed algorithm was trained and tested on the open face dataset CelebA and landscape dataset Places2.The experimental results show that the proposed method can enhance the accuracy of image detail and effectively avoid the restoration distortion.The proposed method is superior to the four classical algorithms used for comparison in visual effect,peak signal-to-noise ratio(PSNR),structural similarity(SSIM) and average error of repair.