基于改进CycleGan模型和区域分割的表情动画合成

Expression Animation Synthesis Based on Improved CycleGan Model and Region Segmentation

针对现有人脸表情合成大多依赖于数据源驱动,且存在生成效率低、真实感差的问题,提出一种基于改进CycleGan模型和区域分割的表情动画合成新方法。新方法可实时地合成新表情动画,且具有较好的稳定性和鲁棒性。所提方法在传统CycleGan模型的循环一致损失函数中构造新的协方差约束条件,可有效避免新表情图像生成时出现的色彩异常和模糊不清等现象;提出分区域训练的思想,用Dlib人脸识别数据库对人脸图像进行关键点检测,通过检测到的关键特征点将源域和目标域的人脸分割成左眼、右眼、嘴部和剩余人脸部分共4个区域块,并利用改进的CycleGan模型对每块区域单独进行训练;最后将训练结果加权融合成最终的新表情图像。分区域训练进一步增强了表情合成的真实感。实验数据来自英国萨里大学的语音视觉情感(SAVEE)数据库,在Tensorflow框架下,用python 3.4软件进行实验结果的展示。实验表明,新方法无需数据源驱动,可直接在源人脸动画序列上实时地生成真实、自然的新表情序列,且对于语音视频可保证新面部表情序列与源音频同步。

Aiming at the problems of mostly relying on data source driver,low generation efficiency and poor authenticity of the existing facial expression synthesis methods,this paper proposes a new method for expression animation synthesis based on the improved CycleGan model and region segmentation.This new method can synthesize new expression in real time and has good stability and robustness.The proposed method constructs a new covariance constraint in the cycle consistent loss function of the traditional CycleGan model,which can effectively avoid color anomalies and image blurring in generation of new expression images.The idea of zonal training is put forward.The Dlib face recognition database is used to detect the key points of the face images.The detected key feature points are used to segment the face in domain source and target domain into four zones:left eye,right eye,mouth and the rest of the face.The improved CycleGan model is used to train each region separately,and finally the training results are weighted and fused into the final new expression image.The zonal training further enhances the authenticity of expression synthesis.The experimental data comes from the SAVEE database,and the experimental results are presented with python 3.4 software under the Tensorflow framework.Experiments show that the new method can directly generate real and natu-ral new expression sequences in real time on the original facial expression sequence without data source driver.Furthermore,for the voice video,it can effectively ensure the synchronization between the generated facial expression sequence and the source audio.