基于深度学习的通用性图像复原方法研究

Universal Image Inpainting Method Based on Deep Learning

目的进一步提高图像复原的性能。方法提出一种基于隐式知识迁移(Implicit knowledge transfer)和显式掩码引导(Explicit mask guide)的图像复原通用方法IECNN。将一般的图像复原任务明确拆分为退化区域定位和区域引导复原等2个阶段。首先利用掩码预测网络中固有的退化定位知识,并进行训练,检测严重退化区域,然后提出一种自适应的注意力知识蒸馏方法,将退化区域知识隐式迁移到复原网络中,且无须任何额外的推理计算,随后提出一种掩码引导下的2种模块,在扩充全局感受野的同时重点关注退化区域,以此显式进行图像复原。结果在进行消融实验时,通过可视化特征图与成对关系图直观展现了各个组件的有效性。为了证明文中方法的通用性,在4种空间变化的图像复原任务中,以峰值信噪比(Peak signal to noise ratio)和结构相似性(Structural similarity)2个指标与其他基准方法进行了定量比较,在视觉效果上进行了定性比较。结论证明了隐式知识迁移和显式掩码引导对于图像复原的有效性。

The work aims to improve the performance of image inpainting.IECNN,an universal image inpainting method based on Implicit Knowledge Transfer and Explicit Mask Guide,was proposed.The proposed universal image inpainting task was clearly divided into two stages of degraded region localization and region guided restoration.Firstly,the degradation localization knowledge inherent in mask prediction network was used and trained to detect the severely degraded region.Then,an adaptive attentional knowledge distillation method was proposed to transfer the degraded region knowledge implicitly into the restoration network without any extra inference cost.Then,two mask guided modules were proposed to extend the global receptive field and focus on the degraded region,so as to explicitly restore the image.In the ablation experiment,the effectiveness of each component was visually demonstrated by visual feature maps and pairwise relationship maps.In order to prove the universality of the proposed method,two indexes,peak signal to noise ratio and structural similarity,were quantitatively compared with other reference methods in four spatially varied image restoration tasks.Then,qualitative comparison was made on visual effect.The results show that implicit knowledge transfer and explicit mask guide are effective for image inpainting.