深度伪造及其取证技术综述

A survey of Deepfake and related digital forensics

深度学习作为机器学习的一个具有前景的重要分支,在计算机视觉方面取得了重大突破。深度伪造(Deepfake)通常指的是使用深度学习(deep learning)进行涉及人脸和人声的多媒体伪造技术,如果被恶意滥用会给社会带来灾难。深度伪造不仅限于面部的替换,还有修改面部特征、修改表情、唇形同步、姿势变换、完整脸生成、篡改音频到视频以及文本到视频等方式。人类面部在社会、政治、经济等方面的敏感性,使得深度伪造技术威胁着社会和个人的安全。对深度伪造产物进行检测也成为数字取证领域的一个重要研究课题。为了提供对Deepfake检测研究工作的最新概述,本文描述了各种针对解决Deepfake相关问题的处理方法。本文主要参考了谷歌学术检索2018—2022共5年的深度伪造论文,分为不同类别进行分析比较,并且详细介绍了深度伪造数据集的特点以及伪造方法,简述了深度伪造技术及其基本原理,介绍了检测器在深度伪造技术数据集上的性能效果,分别从输入维度、浅层特征和深层特针对深度伪造检测技术进行分类,并对未来发展前景进行展望。

Deep learning,a promising branch of machine learning,has made significant breakthroughs in computervision.However,Deepfake,which refers to the set of techniques for forging human-related multimedia data using deeplearning,can bring disasters to society if used maliciously.It is not only limited to facial replacement,but also othermanipulations,such as fabricating facial features,manipulating expressions,synchronizing lips,modifying head gestures,entire face synthesis,and tampering related audios to videos and related texts to videos.Moreover,it can be used to gener⁃ate faked pornographic videos or even faked speeches to subvert state power.Thus,deep forgery technology can greatlythreaten society and individuals,thereyby detecting Deepfake has also become an important research topic in digital foren⁃sics.We conducted a systematic and critical survey to provide an overview of the latest research on Deepfake detection by exploring the recent developments in Deepfake and related forensic techniques.This survey mainly referred to papers onDeepfake in Google Scholar during 2018—2022.This survey divided the Deepfake detection techniques into two categoriesfor analysis and comparison:input dimensions and forensic features.First,a comprehensive and systematic introduction ofdigital forensics is presented from the following aspects:1)the development and security of deep forgery detection technol⁃ogy,2)Deepfake technology architecture,and 3)the prevailing datasets and evaluation metrics.Then,this survey pres⁃ents Deepfake techniques in several categories.Finally,future challenges and development prospects are discussed.Interms of image and video effects,Deepfake techniques are usually divided into four categories:face replacement,lip syn⁃chronization,head puppets,and attribute modification.The most commonly used Deepfake algorithms are based on selfencoders,generative adversarial networks,and diffusion models.An typical autoencoder consists of two convolutional neu⁃ral networks acting as an encoder and a decoder.The encoder reduces the dimensions of the input targets’facial image andencodes it into a vector corresponding to facial features.We share the parameters of the encoder;that is,we use the sameencoder to learn only the common feature information for the encoder network.The structure of a generative adversarial net⁃work is based on a generator and a discriminator.The generator is similar to the decoder in an autoencoder,which convertsthe input noise into a picture and sends it to the discriminator for discrimination along with the real existing picture.Thediscriminator and the generator use back-propagation to optimize the parameters.Moreover,diffusion model is a parameter⁃ized Markov chain trained using variational inference to produce samples that match the data after a finite time.There arealways two processes to train a diffusion model.One is the forward process,also called the diffusion process.The other pro⁃cess is reverse diffusion,also known as the reverse process,which slowly restores the original image from noise throughcontinuous sampling.In the Deepfake detection task,the datasets have also evolved to fill past gaps.In general,this sur⁃vey divides the Deepfake datasets into two generations.The first-generation datasets are often not large enough,and thequality of the content is not satisfying because of the low degree of research fervor.These source videos are usually fromvideo sites or existing face datasets,which can lead to copyright and privacy concerns.The main first-generation datasetsare UADFV,DF-TIMIT,FaceForensics,and diverse fake face dataset(DFFD).The second generation of face forgery data⁃sets has improved forgery effects and image clarity.The main second-generation datasets are Celeb-DF,Deepfake detec⁃tion challenge dataset(DFDC)preview,DeeperForensic-1.0,DFDC,Korean Deepfake detection dataset(KoDF),etc.Interms of input dimension,detecting Deepfake can be roughly divided into three categories:1)the first category is inputtingthe image or key frame from the video,namely,inputting the image or key frame extracted from the video and judging theinput data from the visual performance.This category is commonly used because it can be promoted easily to other com⁃puter vision classification models,and most Deepfake videos are conducted by frame-by-frame images.2)The second isinputting continuous frames from video.In particular,multiple consecutive frames are inputted to allow the model to per⁃ceive the difference in the relationship between the frames from real and fake videos.3)The third is inputting multipleframes and audio simultaneously from the video;that is,the video’s authenticity is detected by examining its video framesand audio together.The features focused on by Deepfake detection techniques also vary.This survey divides them into fourcategories:1)the frequency domain-based approach looks for anomalies in the video at the signal level,treating the videoas a sequence of frames and a synchronized audio signal.Such anomalies,including image mismatches and mismatches inaudio-video synchronization,are usually generated from the mismatches at the signal level during Deepfake video genera⁃tion.2)The texture and spatio-temporal approaches tend to focus only on face position and feature matching in the forgedvideo generation process,where breakdowns that violate the laws of physics and human physiology may occur.3)Thereconstruction–classification learning methods emphasize the common compact representations of genuine faces andenhance the learned representations to be aware of unknown forgery patterns.Classification learning involves mining theessential discrepancy between real and fake images,facilitating the understanding of forgeries.4)Data-driven methods aredetection methods that do not target specific features.However,they use supervised learning to feed real and fake videosinto the model for training.The road to the research on deep forgery techniques and deep forgery detection is still long.Wemust overcome the existing shortcomings and face the challenges of future technological advances.