多模态维度情感预测综述

A Survey of Dimensional Emotion Prediction by Multimodal Cues

维度情感模型通过几个取值连续的维度(如唤醒维、效价维、支配维等)将情感刻画为一个多维信号.与传统的离散情感模型相比,具有表示情感的范围广、能描述情感的演变过程等优点,近年来受到越来越多情感识别研究者的关注.多模态维度情感预测是一项复杂的工程,预测性能受所使用的模态、每个模态的特征提取、信息融合技术、标注人员的标注误差等多方面影响.为了提高多模态维度情感预测的性能,研究者在各个方面都做出了不懈努力.本文综述了维度情感的概念、标注,维度情感预测的性能评价指标以及多模态维度情感预测的研究现状,对比和分析了各种因素对多模态维度情感预测性能的影响,并总结出多模态维度情感预测面临的挑战及发展趋势.

The dimensional emotion model characterizes emotion as a signal in a multi-dimensional space spanned by several continuously valued dimensions (such as arousal,valence,and dominance).Compared with the discrete emotion model,it has the advantages that it can distinguish subtle difference of emotion,can represent evolution of emotion, etc.So the dimensional emotion model has been paid more and more attention in recent years.Dimensional emotion prediction from multimodal cues is a complex task,the prediction performance is influenced by such as modalities used, features extracted from each modality,information fusion technique,annotation errors.In order to improve multimodal dimensional emotion prediction performance,researchers have made persistent efforts in all aspects.In the paper,concept and annotation of dimensional emotion,performance evaluation criteria of dimensional emotion prediction,and research status of multimodal dimensional emotion prediction are reviewed;influences of various factors on emotion prediction performance are analyzed;challenge and development trend of multimodal dimensional emotion prediction are summarized.