面向图像修复取证的U型特征金字塔网络

U-shaped feature pyramid network for image inpainting forensics

图像修复是一种常见的图像篡改手段,而基于深度学习的图像修复方法能生成更复杂的结构乃至新的对象,使得图像修复取证工作更具有挑战性。因此,提出一种端到端的面向图像修复取证的U型特征金字塔网络(FPN)。首先,通过自上而下的VGG16模块进行多尺度特征提取,并利用自下而上的特征金字塔架构对融合后的特征图进行上采样,整体流程形成U型结构;然后,结合全局和局部注意力机制凸显修复痕迹;最后,使用融合损失函数以提高修复区域的预测率。实验结果表明,所提方法在多种深度修复数据集上的平均F1分数和IoU值分别为0.7919和0.7472,与现有的基于扩散的数字图像修复定位(LDI)、基于图像块的深度修复取证方法(Patch-CNN)和基于高通全卷积神经网络(HP-FCN)方法相比,所提方法具有更好的泛化能力,且对JPEG压缩也具有较强的鲁棒性。

Image inpainting is a common method of image tampering.Image inpainting methods based on deep learning can generate more complex structures and even new objects,making image inpainting forensics more challenging.Therefore,an end-to-end U-shaped Feature Pyramid Network(FPN)was proposed for image inpainting forensics.Firstly,multi-scale feature extraction was performed through the from-top-to-down VGG16 module,and then the from-bottom-to-up feature pyramid architecture was used to carry out up-sampling of the fused feature maps,and a U-shaped structure was formed by the overall process.Next,the global and local attention mechanisms were combined to highlight the inpainting traces.Finally,the fusion loss function was used to improve the prediction rate of the repaired area.Experimental results show that the proposed method achieves an average F1-score and Intersection over Union(IoU)value of 0.7919 and 0.7472respectively on various deep inpainting datasets.Compared with the existing Localization of Diffusion-based Inpainting(LDI),Patch-based Convolutional Neural Network(Patch-CNN)and High-Pass Fully Convolutional Network(HP-FCN)methods,the proposed method has better generalization ability,and also has stronger robustness to JPEG compression.