MC-CA:基于模态时序列耦合与交互式多头注意力的多模态情感分析

MC-CA:Multimodal sentiment analysis based on modal temporalcoupling and interactive multi-head attention

针对现有多模态情感分析模型对模态交互问题的忽视导致其性能受限的问题,提出一种基于模态时序列耦合与交互式多头注意力的多模态情感分析模型(MC-CA)。利用仿射变换耦合模态的情感信息和时序信息,使用交互式多头注意力机制获取模态间的交互信息,利用多通道情感预测方法综合全局与局部信息实现多模态协同训练。在多个公开数据集上的实验结果表明,该模型能够建立多模态数据之间的交互,在多模态情感分析任务中取得优异的性能。

Multimodal sentiment analysis aims to comprehensively analyze multimodal data to obtain people’s opinions and attitudes.However,the existing models ignore the problem of multimodal interaction,which leads to their limited performance.Aiming at this problem,this paper proposes a multimodal sentiment analysis model(MC-CA)based on modal temporal coupling and interactive multi-head attention.Firstly,the model utilizes affine transformation to fuse the sentiment information and the timing information.Then the model applies the interactive multi-head attention mechanism to obtain the interaction information among modalities.Finally,the model employs a multi-channel sentiment prediction method to integrate global information and local information to achieve multimodal collaborative training.Experimental results on multiple public datasets show that this model can establish interactions between multimodal data and achieve excellent performance in multimodal sentiment analysis tasks.