基于残差注意力融合和门控信息蒸馏的图像修复

Image Inpainting via Residual Attention Fusion and Gated Information Distillation

图像修复在计算机视觉任务中具有重大的意义和价值。近年来,基于深度学习的图像修复模型被广泛应用于该领域中。但是现有的深度学习图像修复模型存在对破损图像中有效信息的利用不足和受破损图像中掩码信息干扰的问题,从而导致修复出的图像的部分结构缺失和部分细节模糊。为此,文中提出了基于残差注意力融合和门控信息蒸馏的图像修复模型。首先,该图像修复模型由生成器和鉴别器两部分组成,生成器的主干结构采用U-Net网络,由编码器和解码器组成;鉴别器采用马尔可夫鉴别器,由6个卷积层组成。然后,在编码器和解码器中分别构建残差注意力融合结构,以增强破损图像中有效信息的利用和减少掩码信息的干扰。最后,在编码器和解码器的跳跃连接中嵌入门控信息蒸馏模块,进一步提取待修复图像中的有效低级特征。在公开人脸和街景数据集上的实验结果表明:文中模型在语义结构和纹理细节方面具有更好的修复效果;文中模型的结构相似性、峰值信噪比、平均绝对值误差、最小平方误差和弗雷歇距离指标均优于5种对比模型,说明文中模型的修复质量优于对比模型。

Image inpainting is of great significance and value in computer vision tasks.In recent years,image inpainting models based on deep learning have been widely used in this field.However,the existing deep learning image inpainting models have the problems of insufficient utilization of the effective information in the damaged image and interference by the mask information in the damaged image,which leads to the loss of part of the structure and fuzzy part of the details of the repaired image.Therefore,this paper proposed an image inpainting model based on a residual attention fusion and gated information distillation.Firstly,the model consists of two parts,the generator and the discriminator.The backbone structure of the generator uses the U-Net network and consists of two parts,the encoder and the decoder.The discriminator uses a Markov discriminator and consists of six convolutional layers.Then,the residual attention fusion block was used in the encoder and decoder,respectively,to enhance the utilization of valid information in the broken image and reduce the interference of mask information.Finally,a gated information distillation block was embedded in the skip connection of the encoder and decoder to further extract the low-level features in the damaged image.The experimental results on public face and street view datasets show that,the proposed model has better repair performance in semantic structure and texture details;the proposed model outperforms the five contrast models in structural similarity,peak signal to noise ratio,mean absolute error,mean square error and Fréchet distance indicators,demonstrating that the inpainting quality of the proposed model is superior to the compared models.