基于BERT和双通道注意力的文本情感分类模型

Text Sentiment Classification Model Based on BERT and Dual Channel Attention

对于句子级文本情感分析问题,目前的深度学习方法未能充分运用情感词、否定词、程度副词等情感语言资源。提出一种基于变换器的双向编码器表征技术(Bidirectional encoder representations from transformers,BERT)和双通道注意力的新模型。基于双向门控循环单元(BiGRU)神经网络的通道负责提取语义特征,而基于全连接神经网络的通道负责提取情感特征;同时,在两个通道中均引入注意力机制以更好地提取关键信息,并且均采用预训练模型BERT提供词向量,通过BERT依据上下文语境对词向量的动态调整,将真实情感语义嵌入到模型;最后,通过对双通道的语义特征与情感特征进行融合,获取最终语义表达。实验结果表明,相比其他词向量工具,BERT的特征提取能力更强,而情感信息通道和注意力机制增强了模型捕捉情感语义的能力,明显提升了情感分类性能,且在收敛速度和稳定性上更优。

As for sentence-level emotion analysis,current deep learning methods fail to make full use of emotional language resources such as emotion words,negative words and degree adverbs.A new model is proposed based on bidirectional encoder representations from transformers(BERT)and dual channel attention.One channel based on bi-directional GRU(BiGRU)neural network is responsible for extracting semantic features,while the other based on full connection neural network is responsible for extracting emotional features.At the same time,attention mechanism is introduced into both the channels to better extract key information,and the pre-trained model Bert is used to provide word vectors and thereafter adjust them dynamically according to the context so as to embed real emotional semantic into the model.The final semantic expression is obtained through the fusion of semantic features and emotional features from the two channels.The experimental results show that,compared with other word vector tools,BERT has a better feature extraction ability,while the emotional information channel and the attention mechanism enhance the model’s ability to capture emotional semantics,which significantly improves the performance of emotion classification and its convergence speed and stability as well.