融合小波快捷连接生成对抗网络的面部性别伪造

Gender forgery of faces by fusing wavelet shortcut connection generative adversarial network

面部属性编辑领域的主流方法由于数据以及模型架构的限制,存在如下两方面的问题:自编码器模型在编码以及解码阶段的瓶颈结构会丢失特征信息,以及在解码过程中逐阶段对源域特征进行连续的样式注入,使得生成图像中目标域信息占比过大,导致生成图像丢失了源域的身份信息以及部分细节;在人脸图像中,图像人物的性别、种族或年龄等属性的差异会使得不同图像的频域信息组成差别较大,在无监督的训练前提下,当前主流网络框架并不能在样式注入阶段调整源域信息以及目标域信息之间的占比,导致生成图像仍然存在伪影。针对上述问题,基于生成对抗网络和图像到图像的翻译提出一种面部性别伪造模型,即融合快捷小波连接生成对抗网络(WscGAN)。通过对自编码器结构添加快捷连接,将不同编码阶段的输出通过小波变换进行特征级别的分解,再引入通道注意力机制将其进行逐个处理,以此动态改变在解码过程中不同频率的源域特征信息的比例,最终实现面部图像在性别属性上的伪造。为了验证所提模型的有效性,WscGAN分别在CelebA-HQ数据集和FFHQ数据集上进行了实验。实验结果表明,WscGAN在CelebA-HQ以及FFHQ数据集上均领先现有先进模型,其中弗雷歇起始距离分别提升5.4%和19.8%,特征度量相似度分别提升1.8%和4.1%。此外,定性的视觉对比结果充分体现了WscGAN可有效改善面部图像性别属性伪造的效果。

The mainstream methods in the field of facial attribute manipulation had the following two defects due to data and model architecture limitations.First,the bottleneck structure of the autoencoder model results in the loss of feature information,and the traditional method of continuously injected styles to the source domain features during the decoding process makes the generated image too referential to the target domain while losing the identity information and fine-grained details.Second,differences in facial attributes composition between images,such as gender,ethnicity,or age can cause variations in frequency domain information.And the current unsupervised training methods do not automatically adjust the proportion of source and target domain information in the style injection stage,resulting in artifacts in generated images.A facial gender forgery model based on generative adversarial networks and image-to-image translation techniques,namely fused wavelet shortcut connection generative adversarial network(WscGAN),was proposed to address the these issues.Shortcut connections were added to the autoencoder structure,and the outputs of different encoding stages were decomposed at the feature level by wavelet transform.Attention mechanism was employed to process them one by one,to dynamically change the proportion of source domain features at different frequencies in the decoding process.This model could complete forgery of facial images in terms of gender attributes.To verify the effectiveness of the model,it was conducted on the CelebA-HQ dataset and the FFHQ dataset.Compared with the existing optimal models,the method improves the FID and LPIPS indices by 5.4% and 11.2%,and by 1.8% and 6.7%,respectively.Furthermore,the effectiveness of the proposed method in improving the gender attribute conversion of facial images is fully demonstrated by the results based on qualitative visual comparisons.