基于BERT-VGG16的多模态情感分析模型

Multi-modal Sentiment Analysis Model based on BERT-VGG16

针对传统情感分析方法只采用文本数据无法充分挖掘情感信息,且单模态数据包含的信息量有限,不能很好反映真实情感状态等问题,提出一种引入注意力机制的多模态情感分析模型。首先,该模型使用预训练模型BERT和VGG16分别从文本数据和图像数据中提取特征。其次,为提高各模态重要特征权重,特征融合时引入注意力机制,融合后的模型可大幅提升数据信息量。实验结果表明,使用基于BERT-VGG16引入注意力机制的多模态特征融合模型比单模态和其他多模态特征融合模型在情感分析效果上有显著提升。

Aiming at the problems that traditional sentiment analysis methods using only text data cannot fully dig out sentiment information,and the amount of information contained in monomodal data is limited,and cannot reflect the true emotional state well,a multi-modal sentiment analysis that introduces an attention mechanism is proposed model.First,the model uses pre-training models BERT and VGG16 to extract features from text data and image data,respectively.Secondly,in order to increase the weight of important features of each modal,the attention mechanism is introduced during feature fusion,and the fusion model can greatly increase the amount of data information.Experiment results show that the use of a multi-modal feature fusion model based on the BERT-VGG16 that introduces the attention mechanism has a significant improvement in the effect of sentiment analysis compared to single-modal and other multi-modal feature fusion models.