基于双网络及多尺度判决器的图像修复算法

Multi-scale Discriminator Image Inpainting Algorithm Based on Dual Network

为了有效解决修复复杂背景及高分辨率图像时产生的边界扭曲、伪影及训练不稳定的缺陷,提出了一种基于双生成对抗网络及多尺度判决器的图像修复算法。首先,将待修复图像输入基于空洞卷积(dilated convolution)层的内容预测网络,以重构损失和基于生成对抗损失的全局判决器为标准,进行粗修复,得到清晰合理、整体语义一致性的结构。然后,将粗修复结果输入细节修复网络,经空洞卷积路径和感知卷积路径解码和反卷积后,送入3个不同尺度的判决器进行优化,提升修复结果的纹理细粒度。最后,使用3个不同尺度的对抗损失优化网络参数,捕获破损区域的多尺度边缘信息,生成合理、逼真的纹理细节。在公认的图像数据集上对本文算法进行修复实验、双网络修复对比、高分辨率修复对比、目标移除实验、消融实验及客观实验,实验结果表明:本文提出的算法在修复背景复杂图像时,能生成合理的结构和清晰的纹理细节;双网络结构优于单网络结构;修复高分辨率图像时得到的纹理细粒度优于对比算法;将本文提出的算法用于高分辨率目标移除,能得到结构清晰合理、纹理细腻的结果;消融实验验证了提出模块的有效性;本文提出的算法的峰值信噪比、结构相似度、平均l_(1)误差和平均l_(2)误差均优于对比的经典修复算法。总之,本文提出的算法能很好地结合图像的整体语义,增强图像细节的修复精度,有效避免结构纹理错乱、像素重叠、边界扭曲等问题。

In order to effectively solve the defects of boundary distortion,artifacts and training instability when repairing complex background and high-resolution images,an image inpainting algorithm based on dual-generation adversarial network and multi-scale arbiter was proposed.Firstly,the damaged image was inputted to the content prediction network based on the dilated convolution layer,in which the reconstruction loss and the global judge based on the generated confrontation loss were adopted as the standard to perform the rough repair.Subsequently,the rough result was fed into the detail restoration network containing a hollow convolution layer and a contextual attention layer,whose outputs were decoded,deconvolved and sent to a three-scale discriminator.The proposed algorithm was trained and tested on the public image data sets.Finally,the network parameters were optimized by using three different scales of anti-loss functions to capture the multi-scale edge information of damaged area and generate reasonable and realistic texture details.The proposed algorithm was evaluated by the restoration experiments,dual network restoration comparison,high-resolution restoration comparison,target removal experiment,ablation experiment and objective experiment.The experimental results show that the proposed algorithm can generate reasonable structure and clear texture details when repairing images with complex backgrounds.The dual network structure is superior over single network structure.When repairing high-resolution images,the texture granularity obtained is better than that of the comparison algorithms.When the proposed algorithm is applied to high-resolution target removal,the results with clear and reasonable structure and fine texture can be obtained.Ablation experiments verify the effectiveness of the proposed modules.The indicators obtained by the proposed algorithm,i.e.peak signal-to-noise ratio,structural similarity,average error and average error are better than that of the state-of-the-art methods.In summary,the proposed algorithm can combine the overall semantics of the image,enhance the repair accuracy of image details,and effectively avoid structural texture disorder,pixel overlap,boundary distortion.