TSAIE:图像增强文本的多模态情感分析模型

TSAIE: Text Sentiment Analysis Model Based on Image Enhancement

【目的】近年来,以图文结合的多模态数据分析模型已经逐渐成为社交网络中情感分析的重要途径。【方法】本文针对多模态情感分析中存在的图文特征融合问题,提出了一种基于图像增强文本的多模态情感分析模型TSAIE。该模型分别提取文本特征和图片特征,然后设计了基于Transformer编码器与注意力机制的组合注意力图文特征融合模块,通过该模块计算出文本中的每一个词和图片的信息相关度,提升文本特征的情感表征能力,最后将经过组合注意力计算之后的文本特征与图片特征拼接后输入全连接层。【结论】实验结果表明,在MVSA-Single数据集上,情感分类的准确率和F1值分别提高了3.11%和2.53%,在MVSA-Multi数据集上,情感分类的准确率和F1值分别提高了1.33%和0.74%,从而验证了TSAIE模型的有效性。

[Objective]In recent years,the multimodal data analysis model combined with text and image has gradually become an important approach for sentiment analysis in social networks.[Methods]Aiming at the problem of image and text feature fusion in multimodal sentiment analysis,a multimodal sentiment analysis model TSAIE based on image enhancement is proposed.The model extracts text features and image features respectively.A combined attention graphic feature fusion module based on Transformer Encoder and attention mechanism is designed.Through this module,the information relevance of each word and picture in the text is calculated to improve the emotional representation ability of text features.Finally,the text feature and image feature after combined attention calculation are concatenated and input into the full connection layer.[Conclusions]The experimental results show that the accuracy and F1 value of sentiment classification are increased by 3.11%and 2.53%respectively on the MVSA-single data set,and 1.33%and 0.74%respectively on the MVSA-Multi data set,thus verifying the effectiveness of the TSAIE model.