基于帧内-帧间自融合的双流泛化人脸伪造检测方法

Dual-Stream Generalized Face Forgery Detection Method Based on Intra-Inter Frame Self-Blending

现有人脸伪造检测方法往往在已知伪造类型上表现良好,但面对未知数据时检测性能有所下降,模型易受到过拟合的影响,检测泛化性不足。针对此问题,提出一种基于帧内-帧间自融合的双流泛化人脸伪造检测方法,从数据增强和检测器改进2个方面提高检测泛化性。设计帧内-帧间自融合模块,分别利用同帧人脸、帧间人脸进行数据增强:帧内自融合子模块利用同帧人脸生成训练数据,从而避免人脸图像身份信息干扰;帧间自融合子模块利用伪造视频的帧间不一致性,进一步构造多样性丰富、逼真的训练数据集,从而有效防止模型的过拟合,确保检测模型的泛化能力。此外,设计基于通道注意力机制的双流特征融合网络,在网络的浅层提取RGB特征、高频特征并进行融合来挖掘伪造信息,在提升模型性能的同时缓解网络的参数增长。将模型在4个数据集上与9种主流检测方法进行对比实验,结果表明:在跨数据集实验中,所提方法较次优方法AUC均值提高1.52个百分点,EER均值降低1.5个百分点;在跨伪造方法实验中,所提方法在4种伪造方法子数据集上均取得最优或次优效果。实验结果验证了该方法优秀的泛化能力。

Although existing methods for detecting face forgery perform well within familiar source domains,they often suffer from overfitting,leading to a lack of generalizability in face forgery detection.As a result,their performance significantly decreases when faced with unfamiliar or unknown scenarios.To address this issue,this study proposes a dual-stream generalized face forgery detection method based on intra-inter frame self-blending.The intra-inter frame self-blending module is designed to prevent detector overfitting from unrelated identity information by leveraging inconsistencies within and between frames to generate a diverse and realistic forgery training set.This method enhances the generalizability of the detection model.Additionally,a detection model is developed as a dual-stream network using an RGB-frequency feature-enhancing module,which extracts and fuses RGB and high-frequency features within the shallow layers of the network to capture forged artifacts.This method not only enhances the model performance but also alleviates the increase in model parameter size.Experiments are conducted against nine mainstream methods across four datasets,with the proposed model improving the AUC by 1.52%and EER by 1.5%on average in cross-dataset experiments.In addition,it ranks first or second in all four sub-datasets of different manipulations in cross-manipulation experiments.These results demonstrate that the proposed method achieves excellent generalizability in face forgery detection.