基于视频上下文和高维融合的突发事件中网民情感分析研究

Sentiment Analysis of Online Users in the Emergency Based on Video Context and High-dimensional Fusion

【目的/意义】目前有关突发事件中网民情感分析研究多基于文本或文本结合图片的数据,缺乏对视频这一多模态内容的研究。同时在多模态情感分析中,现有文章缺乏对视频上下文关系建模和不同模态特征充分融合相结合的研究。【方法/过程】基于此,本文提出基于视频上下文和高维融合的网民情感分析模型,利用基于双向门循环单元(Bidirectional-Gated Recurrent Units,Bi-GRU)的神经网络学习视频中不同模态的上下文关系,将其两两融合,输入至Bi-GRU网络中,学习不同模态融合后的上下文关系。最后利用三重笛卡尔积的方式充分融合双模态特征,得到高维的三模态融合特征,输入多层全连接层中,从而获得情感类别。并将提出的模型在"新冠疫情"突发事件真实数据集中进行实证研究,同不同模态和基线模型(TFN、HFCM等)进行对比。【结果/结论】实验结果表明,本文提出的模型具有一定的优越性。【创新/局限】提出的模型和研究方法,能够为突发事件中网民情感分析研究提供新的思路。

【Purpose/significance】Research on online users’sentiment analysis in emergency is mostly based on texts or texts combined with images,lacking the research on the multimodal content of videos.At the same time,the existing multi-modal sentiment analysis research mostly focuses on the feature extraction of different modalities,ignoring the context of videos and modal fusion.【Method/process】Based on that,this paper proposes a sentiment analysis model based on the video context and high-dimensional fusion.The model first uses the Bi-GRU-based neural network to learn the context relationships of different modalities in the video,and then merges them in pairs.The fused features are inputted into the Bi-GRU network to learn the context relationships after the fusion of different modalities.Finally,a triple Cartesian product method is used to fuse bimodal features to capture the dynamic interaction between modalities to obtain high-dimensional fused features,which are inputted into multiple fully-connected layers to output labels.【Result/conclusion】The paper conducts empirical research on the proposed model in the real dataset of"COVID19"emergency,and compares it with the baseline models(TFN,HFCM,etc.).Experimental results show that the proposed model has certain advantages.【Innovation/limitation】The model and research method proposed in this paper can provide new ideas for the sentiment of online users in emergencies.