基于卷积神经网络的三维动画表情生成及情感监督方法

3D animation expression generation and emotional supervisionbased on convolutional neural network

针对在线学习,提出一种三维动画表情生成方法,复现学生学习过程中的情感状态,同时对学生进行情感监督。将视频作为输入,通过卷积神经网络识别人脸动作单元,将识别结果作为驱动虚拟替身的数据使其面部发生形变从而生成表情;基于识别结果计算情感指数,进行可视化分析生成情感监督图表。研究得出:用户对提供的表情生成效果和情感指数符合程度的评分均值达到83.97。手动生成的表情同自动生成表情的对比实验证明:效果一致。最终呈现3种可视化结果(动画表情生成效果、情感指数波动图和情感指数分布图),为教师提供了具有学生隐私保护的情感监督和学习状态反馈。

In recent years,online learning has attracted quite a few students by providing free courses from a growing number of universities or institutions.However,one of the main issues in the online education is the difficulty of the educator for detecting learners’emotional states.This paper proposes a method to automatically generate 3D facial expressions to represent students’emotional state during the learning process.At the same time,it can conduct emotional supervision on students aiming at solving the problems of the lack of sensation and privacy protection in online learning.The video is taken as input,and the facial action unit is firstly recognized by the convolutional neural network.To generate 3D facial expressions,this method uses a script to perform monitoring to obtain a real-time action unit sequence parameter,and drive the corresponding animation curve to deform correctly.Three kinds of student-oriented visual feedback are proposed,and provide teachers with the method of emotion supervision and learning state feedback.In this paper,user study and comparative experiment show the effectiveness of the proposed method.