基于双注意力多层特征融合的视觉情感分析

Visual Sentiment Analysis Based on Multi-level Features Fusion of Dual Attention

为获得更具判别性的视觉特征并提升情感分类效果,构建融合双注意力多层特征的视觉情感分析模型。通过卷积神经网络提取图像多通道的多层次特征,根据空间注意力机制对多通道的低层特征赋予空间注意力权重,利用通道注意力机制对多通道的高层特征赋予通道注意力权重,分别强化不同层次的特征表示,将强化后的高层特征和低层特征进行融合,形成用于训练情感分类器的判别性特征。在3个真实数据集TwitterⅠ、TwitterⅡ和EmotionROI上进行对比实验,结果表明,该模型的分类准确率分别达到79.83%、78.25%和49.34%,有效提升了社交媒体视觉情感分析的效果。

In order to obtain more discriminative image features and improve the accuracy of sentiment classification,a visual sentiment analysis model based on multi-level feature fusion of dual attention is proposed.Firstly,the model extracts multi-channel and multi-level features of images with Convolution Neural Network(CNN).Secondly,the extracted low-level features are given spatial attention weight with spatial attention mechanism,and the extracted high-level features are given channel attention weight with channel attention mechanism to saparately enhance the low and high level features.And finally,the enhaned high-level features and low-level features are fused to form discriminant features for training sentiment classifiers.Experiments on three real datasets TwitterⅠ,TwitterⅡand EmotionROI showed that the classification accuracy of the method reached 79.83%,78.25%and 49.34%,which improved the effect of social media visual sentiment analysis.