带虚拟边约束的面部表情基生成方法

Facial Blendshape Generation Method with Virtual Edge Constraints

针对网格形变迁移算法在生成面部表情基时可能出现的模型表面穿透问题,本文提出一种带虚拟边约束的面部表情基生成算法.该算法基于一组模板表情基,可以为不同的对象目标生成个性化的人脸表情基.首先,算法依据顶点之间的距离信息检测出人脸模型中可能发生模型表面穿透的区域,在这些区域添加新的顶点连接关系,本文称为虚拟边;然后以这些虚拟边约束网格形变迁移算法,在生成表情基的同时防止模型发生表面穿透现象.在不同人脸模型上的实验结果表明,算法添加虚拟边的精度和效率优于手动操作以及之前提出的相关算法,并且虚拟边约束可以有效防止表情迁移过程中的模型穿透问题.最后,本文实验为不同的人脸网格生成了表情基,并应用生成的表情基进行表情迁移,实验结果验证了本文提出的带虚拟边约束的面部表情基生成方法的有效性.

There is an increasing need for creating personalized digital avatar as the explosive develop-ment of AR and Metaverse applications.Blendshape based facial animation is a de-fact standard technique for expression synthesis of the digital avatars.A facial blendshape consists of a set of models corresponding to the typical semantic expressions.However,the blendshape is traditionally created by experienced artists using professional animation tools,which is time consuming and cost expensive.In this paper,we proposed an automatic facial blendshape generation method,which takes as input a set of template blendshapes and transfer the blendshape template to target subjects.It processes the model in the template blendshape one by one,and takes advantages of the deformation transfer technique to generate the personalized expression model for the target subject.Model penetration is a common problem incurred in the blendshape transfer process.To address this problem,we propose virtual edge constraints to prevent the penetration for the de-formation transfer equation.It is noticed that the penetration is the result of self-intersection and usually occurs in the thin parts with two or more layers of surfaces,e.g.mouth,eye,and ear.To automatically de-tect the parts that are prone to penetration,we examine the geodesic distance and the Euclidean distance between the vertices of the target face model.If a pair of vertices have long geodesic distance but short Euclidean distance,then the corresponding area has high penetrating possibility.For such vertex pair,we create a virtual edge,and add a soft length constraint to preserve the length of the virtual edge with the aim to prevent penetration.The length constraint is implemented as preserving the transformed edge vector.We use two threshold values to control the distribution of the virtual edges.One threshold is on the Euclidian distance,by which we excludes the distant vertex pairs.The other threshold is on the geodesic-Euclidean distance ratio,by which we exclude the vertex pairs that have low geodesic-Euclidean distance ratio.For detailed face models with densely sampled vertices,there will be a large number of virtual edges connect-ing a given vertex,which maybe redundant for anti-penetration purpose.In this case,we keep 3-5 shortest virtual edges but discard the other longer ones.Once a virtual edge is formed,the other vertices near the edge ends are excluded for later virtual edge selection,in order to further reduce the number of the virtual edges.The modified optimization energy function for facial blendshape generation consists of two parts,one of which is the deformation transfer energy,and the other is virtual edge preservation energy.The vir-tual edge energy is weighted by a super parameter.In this way,both the inner surface and the outer surface in the multilayer parts are constrained to deform accordingly to avoid penetrate each other.We conduct a number of experiments to demonstrate the validity of the proposed virtual edge constraints in our facial blendshape generation methods.Experiments on different subjects,including scanned and designed models,show that the method can efficiently detect potential penetration areas and properly constructthe virtual edge constraints.When generating blendshapes with complex facial expressions,such as cheek contraction and pout,the proposed virtual edge constraints can effectively maintain the inner structural features and avoid the surface penetration.Compared to the previous volumetric deformation methods,the proposed method demonstrated advantages in terms of efficiency and the ability to handle complex meshes.Finally,we show a combination of the virtual edge constraints with the example based facial rigging method,which demonstrates that the application ability of our virtual edge constraint method.