指令驱动的虚拟角色风格化动作生成

Instruction-Driven Styled Motion Generation for Virtual Characters

复杂场景下,指令驱动的动作生成任务存在着指令理解不准确、生成的动作与指令任务不相关等挑战.为了解决这个问题,提出了一种方法框架,通过结合指令和场景信息,风格化生成虚拟角色的动作.该方法框架分为指令解析和动作生成两个部分.方法首先预定义了一组有限的原子动作集合,在指令解析部分运用大模型,将文本指令解析成由原子动作组成的子任务,在动作生成部分,基于条件变分自编码器(conditional variational autoencoder,cVAE)设计了一个逐帧动作生成网络.在动作生成阶段,考虑不同的风格特征,例如角色的高矮胖瘦等属性和文本指令描述的开心、难过等特征,实现风格化动作生成任务.在卧室、公园、客厅和厨房等4个场景中进行了定性实验和用户研究,证明了方法的有效性、动作的真实性和风格的多样性.

To solve the problems existed in instruction-driven motion generation tasks,such as inaccurate in-struction comprehension and unrelated motion generation between the given task and the results,under complex scenes,a novel method was proposed based on the combination of instructions and scene information to gener-ate the styled motions for virtual characters.Two main components were arranged in the new method,including instruction parsing and motion generation.Firstly,using a large language model,a set of limited atomic actions were predefined to parse textual instructions into subtasks with the atomic actions in the instruction parsing stage.And then,in the motion generation stage,based on a conditional variational autoencoder(cVAE)to design a frame-by-frame motion generation network,the instruction-driven styled motion generation tasks were carried out,considering various style features such as age(e.g.,young,old)and style features described in the textual in-structions(e.g.,happy,sad).Finally,consumer investigation and four determine tests were carried out for bed-room,park,living room and cookroom scenes,validating the method efficacy,the motion authenticity and style diversity.