基于双线性融合的多模态细粒度情感分析

Multimodal Fine-grained Sentiment Analysis Based on Bilinear Fusion

为充分挖掘多模态情感分析中不同模态内部消息及其复杂的交互关系,消除噪声干扰,最大化发挥数据融合的优势,提出一种基于双线性融合的多模态细粒度情感分析方法,通过两个独立的特征,提取模块对单个模态特征进行编码,充分挖掘模态内部信息,利用双线性融合方法获取模态间的交互信息,得到融合后的特征向量,通过Softmax层进行细粒度的情感分类。与单模态情感分析和一般的多模态情感分析相比,实验取得了很好的结果。

In order to fully explore the internal messages of different modes and the complex interaction relationships between them in multimodal sentiment analysis,eliminate noise interference,and maximize the advantages of data fusion,the study proposes a multimodal fine-grained sentiment analysis method based on bilinear fusion.This method first codes a single modal feature through two independent feature extraction modules to fully excavate the modal internal information.Then the interaction information between the modes is obtained by the bilinear fusion method to obtain the feature vector after fusion.Finally,fine-grained emotion classification is performed by the Softmax layer.Compared with singlemodal sentiment analysis and general multimodal sentiment analysis,the experiment has achieves good results.