面向多方对话的中文多模态情感语料库构建

Construction of Chinese multi-modal emotion corpus for multi-party dialogue

针对多方对话中文多模态情感语料库资源不足的情况,构建了一个面向中文的多模态多方对话情感语料库:确立了对话情感/情绪标注体系,收集并整理了电视剧《天真派武林外传》中的数据,以话语为单位切分出每条话语对应的文本、声音和视频,标注了330段对话,共计5541条话语;设计了一个多模态多方对话情感分析模型,并在所构建语料库上开展了实验。实验结果表明:单模态情感分析的效果中,文本模态好于声音模态和视频模态;多模态情感分析的综合效果比单模态文本分析的效果差,但好于声音和视频分析的效果。

In view of the insufficient resources of Chinese multi-modal emotion corpus for multi-party dialogue,a corpus was constructed.The dialogue sentiment/emotion annotation system was established,and the data in the TV drama Innocent School:My Own Swordsman,was collected and sorted out.The text,sound,and video corresponding to each discourse were divided by utterance,and 330 dialogues with 5541 utterances were annotated with the sentiment tag and emotion tag.Finally,a baseline model for multi-modal sentiment analysis was designed for multi-party dialogue.The experimental results show that the text mode is better than the sound mode and video mode in the effect of single modal sentiment analysis.The comprehensive effect of multi-modal sentiment analysis is lower than that of text modal but higher than that of the other two,respetively.