基于表情和语音的多模态情感识别研究

Multi-modal Emotion Recognition Based on Video and Audio

由于单一特征的局限性,单一模态的情感识别研究往往由于含有的有效信息量较少或含有的噪声信息过多而导致识别结果与实际情况有着较大的差异。而不同类型的输入特征,相对于单一特征而言,包含着充分的、互补的情感信息。因此,本研究基于eNTERFACE数据库,提取了SIFT特征作为表情特征数据以及使用openSMILE工具包提取的1 582维声学及统计特征作为语音特征数据,分别运用支持向量机SVM和稀疏表示SR方法进行情感识别。最后采用决策层融合的方式,在该数据库上获得了比较好的效果。

Single modality is usually far from satisfactory due to insufficient data or overmuch noi ferent sensors may carry redundant, complementary information, and lead to improve the performance. fore we use eNTERFACE database, extract the SIFT feature as the face emotion feature and using open tools extract 1582 dimension speech feature, and classify by the SVM and SR. Finally we fuse the mu on the score level, and achieve the best recognition results. se. Dif- There- SMILE ltimodal