融合多尺度梯度特征的人脸素描照片合成

Face Sketch-photo Synthesis Based on Multi-scale Gradient Features

人脸素描照片合成是异质图像变换重要分支,近年来受到广泛关注,在数字娱乐和执法领域都得到了广泛应用。近几年基于生成对抗网络的方法在图像跨域转换方面取得了较大的进步,但合成图像会产生噪声、伪影等问题。以U-Net网络为基础,提出了一种融合多尺度梯度特征的人脸素描照片合成方法,该方法结合了MSG-GAN的思想:允许判别器不仅使用生成器最终输出的梯度,而且还使用从上采样中间层输出的不同分辨率的梯度。同时在U-Net网络中加入了残差学习单元,来缓解深度神经网络训练过程中产生的梯度消失和梯度爆炸的问题。此外还加入了基于MRF-CNN的伪人脸特征生成器,采用块匹配的方法来生成伪人脸图像用于监督生成器的人脸生成。最后在CUFS和CUFSF数据集上的实验结果表明所提出方法的有效性。

Face sketch-photo synthesis is an important branch of heterogeneous image transformation.It has been widely used in digital entertainment and law enforcement in recent years.In recent years,the methods based on generative adversarial network have made great progress in image cross-domain transformation,but the synthetic image will produce noise,artifact.Therefore,based on U-Net network,this paper proposes a face sketch-photo synthesis method that integrates multi-scale gradient features.This method combines the idea of MSG-GAN:it allows the discriminator to use not only the gradient output of the generator,but also the gradient output of different resolutions from the upper sampling middle layer.At the same time,the residual learning unit is added to the U-Net network to alleviate the problems of gradient disappearance and gradient explosion in the training process of deep neural network.In addition,the pseudo face feature generator based on MRF-CNN is added,which uses the block matching method to generate the pseudo face image,which is used to supervise the face generation of the generator.Finally,the experimental results on CUFS and CUFSF datasets show the effectiveness of the proposed method.