基于图卷积的手势骨架生成

Skeleton-based gesture generation based on Graph Convolutional Networks

目前手势生成的工作多用于从语音或文本中产生协同的手势以及实现手势数据增强。前者作为非语言信号辅助交流,却难以单独表达语义。对于后者,大多数都是将骨骼关节点当作图像的一个像素,整体当作图像处理,而没有考虑到关节点间丰富的人体结构信息,从而可能导致生成的结果是扭曲的、不自然的。本文提出了基于图卷积的生成式模型,以有效地编码结构信息到手势生成中。研究中将本文的方法与基于全连接神经网络以及基于卷积神经网络的方法进行了对比,实验结果表明,本文生成的手势在定量和定性结果上有了明显的改善。图卷积在手势骨架生成上的成功应用,可以进一步指导手势骨架到真实手势的生成工作,因而对生成自然、真实的手势有重要意义。

Currently,the work of gesture generation is mostly to generate coordinated gestures from speech or text and realize the data augmentation for gesture.The former is used as nonverbal signals to conduce communication,but it is difficult to express semantics alone.The latter is that in most cases,the skeleton joints are regarded as pixels of the image and a frame of the gesture as an image.However,those do not take the rich structure information among joints into consideration,so it may cause the generated result to be distorted and unnatural.The paper proposes a generative model based on Graph Convolutional Networks to efficiently encode structural information into gesture generation.The research compares the proposed method with methods based on Fully Connected Neural Network and Convolutional Neural Network.The results show that the gestures generated by the proposed method have been significantly improved in quantitative and qualitative results.The successful application of Graph Convolutional Networks in the generation of skeleton-based gestures can further guide the work of generating real gestures from skeleton-based gestures,and it is of great significance for generating natural and real gestures.