结合文本监督和多粒度特征的姿态情绪识别

Posture‑Based Emotion Recognition via Text‑Supervision and Multi‑granularity Features

通过分析人体3D骨架的连续运动规律,建模姿态情绪识别方法,在人机交互等领域具有广泛的应用前景。针对姿态特征易混淆导致的骨架局部情绪性特征难捕获问题,提出了一种结合文本监督和多粒度特征的姿态情绪识别方法。通过多粒度特征融合网络模块,对情绪识别问题中人体的动态姿态进行建模。同时,采用文本化描述的标签对融合情绪特征进行监督训练,引导网络提取骨架各个部位的情绪性特征。在MPI情绪身体表达数据集上对该方法进行评估,试验结果表明,该方法准确率达80.50%,验证了其有效性。

By analyzing the continuous motion patterns of the 3D pose skeleton,modeling a posture-based emotion recognition method has a wide application prospect in human-computer interaction.Aiming at the difficulty of capturing the local emotional features caused by easily-confused posture fea-tures,a posture-based emotion recognition method via text-supervision and multi-granularity features is proposed.With multi-granularity feature fusion network module,the dynamic posture of the human body in the emotion recognition problem is modeled.Meanwhile,the text description of the label is used to supervise and train the fused emotional features,guiding the network to extract the emotional features of each skeleton part.The method is evaluated on the MPI emotional body expression datas-et.The experimental results show that the accuracy of the method reaches 80.50%.Thus,its effec-tiveness is verified.