Deep generative models allow the synthesis of realistic human faces from freehand sketches or semantic maps.However,although they are flexible,sketches and semantic maps provide too much freedom for manipulation,and t...Deep generative models allow the synthesis of realistic human faces from freehand sketches or semantic maps.However,although they are flexible,sketches and semantic maps provide too much freedom for manipulation,and thus,are not easy for novice users to control.In this study,we present DeepFaceReshaping,a novel landmarkbased deep generative framework for interactive face reshaping.To edit the shape of a face realistically by manipulating a small number of face landmarks,we employ neural shape deformation to reshape individual face components.Furthermore,we propose a novel Transformer-based partial refinement network to synthesize the reshaped face components conditioned on the edited landmarks,and fuse the components to generate the entire face using a local-to-global approach.In this manner,we limit possible reshaping effects within a feasible component-based face space.Thus,our interface is intuitive even for novice users,asconfirmed by auser study.Our experiments demonstrate that our method outperforms traditional warping-based approaches and recent deep generative techniques.展开更多
基金supported by grants from the Open Researchh Projects of Zhejiang Lab(No.2021KE0AB06)the National Natural Science Foundation of China(Nos.62061136007 and 62102403)+1 种基金the Beijing Municipal Natural Science Foundation for Distinguished Young Scholars(No.JQ21013)d the Open Project Program of the State Key Laboratory of Virtual Reality Technology and Systems,Beihang University(No.VRLAB2022C07).
文摘Deep generative models allow the synthesis of realistic human faces from freehand sketches or semantic maps.However,although they are flexible,sketches and semantic maps provide too much freedom for manipulation,and thus,are not easy for novice users to control.In this study,we present DeepFaceReshaping,a novel landmarkbased deep generative framework for interactive face reshaping.To edit the shape of a face realistically by manipulating a small number of face landmarks,we employ neural shape deformation to reshape individual face components.Furthermore,we propose a novel Transformer-based partial refinement network to synthesize the reshaped face components conditioned on the edited landmarks,and fuse the components to generate the entire face using a local-to-global approach.In this manner,we limit possible reshaping effects within a feasible component-based face space.Thus,our interface is intuitive even for novice users,asconfirmed by auser study.Our experiments demonstrate that our method outperforms traditional warping-based approaches and recent deep generative techniques.
