基于融合策略的突发公共卫生事件网络舆情多模态负面情感识别

Multimodal Negative Sentiment Recognition in Online Public Opinion during Public Health Emergencies Based on Fusion Strategy

突发公共卫生事件以社交媒体为阵地进行线下舆情的线上映射,而图文并茂的多模态信息成为公众情感表达的主要方式。为充分利用不同模态间的关联性和互补性,提升突发公共卫生事件网络舆情多模态负面情感识别精准度,本文构建了两阶段混合融合策略驱动的多模态细粒度负面情感识别模型(two-stage,hybrid fusion strategy-driven multimodal fine-grained negative sentiment recognition model,THFMFNSR)。该模型包括多模态特征表示、特征融合、分类器和决策融合4个部分。本文通过收集新浪微博新冠肺炎的相关图文数据,验证了该模型的有效性,并抽取了最佳情感决策融合规则和分类器配置。研究结果表明,相比于文本、图像、图文特征融合的最优识别模型,本文模型在情感识别方面精确率分别提高了14.48%、12.92%、2.24%;在细粒度负面情感识别方面,精确率分别提高了22.73%、10.85%、3.34%。通过该多模态细粒度负面情感识别模型可感知舆情态势,从而辅助公共卫生部门和舆情管控部门决策。

Social media is used for the online mapping of the offline public opinion on public health emergencies,and multimodal information with image and text becomes the primary means of public sentiment expression.To fully use the correlation and complementarity among different modalities and improve the accuracy of the multimodal negative sentiment recognition in the online public opinion during public health emergencies,this study constructs a two-stage,hybrid fusion strategy-driven multimodal fine-grained negative sentiment recognition model(THFMFNSR)comprising four parts:multimodal feature representation,feature fusion,a classifier,and decision fusion.By collecting image-text data related to COVID-19 from Sina Weibo,this study verifies the effectiveness of the model and extracts the best sentiment decision fusion rules and classifier configurations.The results show that compared with the optimal recognition model with text,image,and image-text feature fusion,the precision of this model in sentiment recognition improved by 14.48%,12.92%,and 2.24%,respectively,and in fine-grained negative sentiment recognition,the precision improved by 22.73%,10.85%,and 3.34%,respectively.The multimodal fine-grained negative sentiment recognition model can sense public opinion situations and assist public health departments and public opinion control departments in decision making.