融合局部语义信息的多模态舆情分析模型

Multimodal Public Sentiment Analysis Model Based on Local Semantic Information

舆情分析被广泛用于事件监测、信息预测、公众意见监测等信息安全相关领域.随着社交媒体的快速发展,Twitter、Facebook和新浪微博等社交网络已成为最主要的信息生成和传播渠道,其中包含着大量带有情绪色彩的图像与文本,对其充分挖掘可以令人更好地理解大众的观点和立场,因此被广泛用作舆论分析的数据来源.许多已有的方法直接从图像中提取特征,作为多模态分析的附加信息,这样往往容易忽略存在于图像局部的高维语义信息,例如表情、动作等.为解决上述问题,提出了一种能够结合局部语义信息的特征提取框架,以及融合视觉与文本特征的多模态情感分析方法.该方法采用图像描述的方法提取图像特征,采用多层卷积的方式提取文本特征,最后训练分类器结合这2种特征进行决策.在情感分析领域的公开数据集MAVA上的实验结果表明,该模型能够有效地挖掘图文特征,在情感分析任务中具有更好的性能.

Public sentiment analysis is widely used in information security fields,such as event monitoring,information forecasting,and public opinion monitoring.With development of social media,social networks such as Twitter,Facebook and Sina Weibo become one of the most important channels for information generation and dissemination,containing a large number of images and texts with emotional colors.Since data in social networks is important of understanding public's views and positions,which is widely used as a source of data for public opinion analysis.Many existing methods extract features directly from images as additional information for multimodal analysis,which tends to neglect high-dimensional semantic information,such as expressions,actions,etc.,that exist locally in the image.To solve these problems,this paper proposes a feature extraction framework that combines local semantic information and a multimodal sentiment analysis method that combines visual and textual features.We use image description method to extract image features,extract text features by multi-layer convolution,and finally train the classifier to combine these two features for decision making.The experimental results on the public data set MAVA in the field of sentiment analysis show that the model can effectively capture the graphic features and have better performance in the sentiment analysis task.