融合背景上下文特征的视觉情感识别与预测方法

Visual emotion recognition and prediction based onfusion of background contextual features

为解决基于视觉的情感识别无法捕捉人物所处环境和与周围人物互动对情感识别的影响、单一情感种类无法更丰富地描述人物情感、无法对未来情感进行合理预测的问题,提出了融合背景上下文特征的视觉情感识别与预测方法。该方法由融合背景上下文特征的情感识别模型(Context-ER)和基于GRU与Valence-Arousal连续情感维度的情感预测模型(GRU-mapVA)组成。Context-ER同时综合了面部表情、身体姿态和背景上下文(所处环境、与周围人物互动行为)特征,进行26种离散情感类别的多标签分类和3个连续情感维度的回归。GRU-mapVA根据所提映射规则,将Valence-Arousal的预测值投影到改进的Valence-Arousal模型上,使得情感预测类间差异更为明显。Context-ER在Emotic数据集上进行了测试,结果表明识别情感的平均精确率比现有方法提高4%以上;GRU-mapVA在三段视频样本上进行了测试,结果表明情感预测效果相较于现有方法有很大提升。

To address the problems of inability to capture the impact of environmental factors and interaction with surrounding individuals on emotion recognition in vision-based affective computing,limitations of describing emotions with a single category,and inability to predict future emotions,this paper proposed a visual emotion recognition and prediction method integrating background context features.This method consisted of an emotion recognition model that integrated background context features(Context-ER)and an emotion prediction model based on GRU and continuous emotion dimensions of Valence-Arousal(GRU mapVA).Context-ER combined facial expressions,body posture,and background context(environment,interaction behavior with surrounding people)features to perform multi-label classification for 26 discrete emotion categories and regression for 3 continuous emotion dimensions.GRU mapVA projected the predicted values of Valence-Arousal onto the improved Valence-Arousal model based on the proposed mapping rules,making the differences between sentiment prediction classes more pronounced.Context-ER was tested on the Emotic dataset,and the results show an average precision improvement of over 4%compared to the state-of-the-art methods.GRU-mapVA was tested on three video samples,and the results demonstrate a signi-ficant improvement in emotion prediction compared to existing methods.