基于门控卷积与注意迁移的二阶图像修复

Second-order image restoration based on gated convolution and attention transfer

针对现有修复算法在处理较大面积缺失时易产生伪影且与原图像语义不符的问题,提出了基于门控卷积与注意迁移的二阶图像修复方法,通过加强待修复图像内部语义对修复网络的影响来确保修复结果整体语义的一致性。首先使用多层卷积对缺损图像进行粗略修复;然后将粗略修复结果输入改进的细化修复网络,使用门控卷积和注意迁移网络对图像内部纹理细节进行修复处理,在编解码阶段引入SimAM模块作为注意力机制,强化对待修复图像中重要信息的筛选能力;最后通过谱归一化的马尔科夫判别器判别真伪同时提供对抗损失,将感知损失与多尺度结构相似性损失作为重建损失再将其与对抗损失相结合作为损失函数。与其他图像修复方法的对比实验表明,本文方法较其中最好结果在结构相似性上提升1.47%,峰值信噪比上提升5.48%。本文方法修复结果更加真实自然且在各种尺寸缺失下均实现了理想的修复效果。

To address the issue that existing restoration algorithms are prone to artifacts when dealing with large areas missing and inconsistent with the semantics of the original image,a second-order image restoration method based on gated convolution and attention transfer is proposed.The overall semantic consistency of the repair results is ensured by strengthening the influence of the internal semantics of the image to be repaired on the repair network.The rough repair results are then input into the improved refinement repair network,and the gated convolution and attention transfer network are used to repair the image’s internal texture details.The SimAM module is introduced as the attention mechanism in the encoding and decoding processes.Finally,the spectrum normalized Markov discriminator is used to determine authenticity while also providing the confrontation loss.The perceived loss and similarity loss of multiscale structure are considered as the reconstruction loss and then combined as the loss function.The comparative experiments with other image restoration methods show that the proposed method improves the structural similarity by 1.47%and the peak signal-to-noise ratio by 5.48%compared with the best results.The repair results of this method are more realistic and natural,and the ideal repair effect is achieved under various sizes.