跨模态交互信息的双模态情感识别

Bimodal Emotion Recognition for Cross-modal Interaction Information

为了充分学习文本和语音两种模态之间的交互信息,本文提出一种基于跨模态交互信息的双模态情感识别方法。该方法首先基于深度学习模型对文本和语音单模态信息进行学习,并采用信息交互的方式进一步得到各模态的上下文信息。最后,在情感数据集中进行实验,通过对比单模态和双模态情感识别模型,该模型在各评价指标上表现出更好的情感分类性能。

In order to fully learn the interaction information between the text and speech modalities, a bimodal emotion recognition method based on cross-modal interaction information is proposed. The method first learns unimodal information about text and speech based on a deep learning model, and uses information interaction to further obtain contextual information about each modality. Finally, experiments are conducted on the sentiment dataset, and by comparing the unimodal and bimodal sentiment recognition models, the model shows better performance in sentiment classification on each evaluation metric.