基于辅助任务和Transformer的人脸正面化网络

Face Frontalization Network Based on Auxiliary Task and Transformer

现有人脸正面化方法仅使用侧面图像生成正面图像,容易带来生成效果不佳及过拟合等问题。对此,提出一种具有辅助任务及Transformer的生成对抗网络(Auxiliary Task Generative Adversarial Network,AT-GAN)。AT-GAN利用多任务的相关性提高人脸正面化效果及泛化性,主任务为人脸正面化本身,使用侧面人脸生成对应的正面人脸;次任务为侧面人像草图生成对应的正面人像草图,引导并辅助主任务,加速网络收敛。两任务之间共享网络权重,并使用基于视觉Transformer的特征交互模块将两部分特征深度融合,提高网络整体的性能,生成更加具有真实感的正面图像。AT-GAN由生成器及判别器组成,生成器的特征提取部分将人脸关键点与空间注意力结合,确保模型准确地提取关键特征。实验结果表明,AT-GAN在MASFD与CAS-PEAL-R1数据集上的Rank-1识别率分别平均提高了4.42%与1.30%,视觉效果及模型泛化性得到提升。

The existing face frontalization methods only use profile images to generate frontal images,which can lead to problems such as poor generation results and overfitting.In this regard,an auxiliary task generative adversarial network(AT-GAN)with Transformer was proposed.AT-GAN used multi-task correlation to improve the effect of face frontalization and generalization.The main task was face frontalization,and the corresponding frontal faces were generated by using profile faces;the secondary task was to generate corresponding frontal portraits from profile portrait sketches,and to guide and assist the main task,so as to accelerate the convergence of network.The network weights were shared by two tasks,and the feature interaction module based on the visual Transformer was used to deeply integrate the two parts of the features,so as to improve the overall performance of the network and generated the more real frontal portraits.AT-GAN consisted of a generator and a discriminator.The feature extraction part combined key points of face with spatial attention to ensure the accurate extraction of key features by the model.The experimental results show that the Rank-1 recognition rate of AT-GAN on the MASFD and CAS-PEAL-R1 datasets is respectively increased 4.42%and 1.30%,and the visual effect and model generalization are improved.