The main challenges in face swapping are the preservation and adaptive superimposition of attributes of two images.In this study,the Face Swapping Attention Network(FSA-Net)is proposed to generate photoreal-istic face...The main challenges in face swapping are the preservation and adaptive superimposition of attributes of two images.In this study,the Face Swapping Attention Network(FSA-Net)is proposed to generate photoreal-istic face swapping.The existing face-swapping methods ignore the blending attributes or mismatch the facial keypoint(cheek,mouth,eye,nose,etc.),which causes artifacts and makes the generated face silhouette non-realistic.To address this problem,a novel reinforced multi-aware attention module,referred to as RMAA,is proposed for handling facial fusion and expression occlusion flaws.The framework includes two stages.In the first stage,a novel attribute encoder is proposed to extract multiple levels of target face attributes and integrate identities and attributes when synthesizing swapped faces.In the second stage,a novel Stochastic Error Refinement(SRE)module is designed to solve the problem of facial occlusion,which is used to repair occlusion regions in a semi-supervised way without any post-processing.The proposed method is then compared with the current state-of-the-art methods.The obtained results demonstrate the qualitative and quantitative outperformance of the proposed method.More details are provided at the footnote link and at https://sites.google.com/view/fsa-net-official.展开更多
基金supported by the National Natural Science Foundation of China(No.61772179)the Hunan Provincial Natural Science Foundation of China(No.2020JJ4152,No.2022JJ50016)+2 种基金the science and technology innovation Program of Hunan Province(No.2016TP1020)the Scientific Research Fund of Hunan Provincial Education Department(No.21B0649)the Double First-Class University Project of Hunan Province(Xiangjiaotong[2018]469).
文摘The main challenges in face swapping are the preservation and adaptive superimposition of attributes of two images.In this study,the Face Swapping Attention Network(FSA-Net)is proposed to generate photoreal-istic face swapping.The existing face-swapping methods ignore the blending attributes or mismatch the facial keypoint(cheek,mouth,eye,nose,etc.),which causes artifacts and makes the generated face silhouette non-realistic.To address this problem,a novel reinforced multi-aware attention module,referred to as RMAA,is proposed for handling facial fusion and expression occlusion flaws.The framework includes two stages.In the first stage,a novel attribute encoder is proposed to extract multiple levels of target face attributes and integrate identities and attributes when synthesizing swapped faces.In the second stage,a novel Stochastic Error Refinement(SRE)module is designed to solve the problem of facial occlusion,which is used to repair occlusion regions in a semi-supervised way without any post-processing.The proposed method is then compared with the current state-of-the-art methods.The obtained results demonstrate the qualitative and quantitative outperformance of the proposed method.More details are provided at the footnote link and at https://sites.google.com/view/fsa-net-official.
