融合多级属性与注意力机制的人脸替换方法

Face Swapping Method Integrating Multi-Level Attributes and Attention Mechanism

针对当前人脸替换方法仅利用目标图像的姿势和表情指导换脸过程中常常忽略背景、光照等其他属性,同时生成的替换人脸与目标图像的融合效果较差的问题,提出一种融合多级属性和注意力机制的人脸替换方法.在提取目标图像属性阶段,基于U-Net结构设计一种多级属性编码器,采用多层次级联的卷积块和反卷积块以及层间连接准确、全面地提取目标图像的表情与背景属性,保留更多细节信息;在生成替换人脸阶段,设计一种结合注意力机制的生成器,利用注意力模型权重自适应地调整源人脸特征和目标属性集成的有效区域,使生成器生成更加符合视觉机制的替换人脸.在FaceForensics++图像集上的实验结果表明,与DeepFaceLab方法相比,该方法所生成的替换人脸与目标图像的结构相似度提高了6.73%,头部姿势差异和面部表情差异分别降低了1.026和0.491.该方法不仅更好地保留了源人脸特征信息,还更大程度地忠实于目标图像属性,达到了良好的替换效果.

The face swapping methods available only use the posture and expression of the target images to guide the face swapping process,which often ignores other attributes such as background and lighting.Moreover,the swapped faces are poorly fused with the target images.Thus,a face swapping method combining multi-level attributes and attention mechanism is proposed.At the stage of extracting target attributes,a multi-level attribute encoder is designed based on the U-Net structure,using multi-level cascaded convolutional and deconvolutional blocks and inter-layer connections to accurately and comprehensively extract the expression and background attributes of the target images,preserving more detailed information.Meanwhile,in the swapped faces generation stage,a generator is designed that incorporates the attention mechanism to adaptively adjust the effective area of the integration of source face features and target attributes,so that the swapped faces are more consistent with visual mechanism.Experimental results on FaceForensics++show that compared with the DeepFaceLab method,the structural similarity between the swapped faces and the target images is improved by 6.73%,and the differences in head posture and facial expression are reduced by 1.026 and 0.491,respectively.Proposed method retains the source face features better,preserves a greater degree of fidelity to the target image attributes,and achieves a good swapped effect.