基于BERT与Loc-Attention的文本情感分析模型

Text sentiment analysis model based on BERT and Loc-Attention

传统的情感分析方法由于没有关注文本相对于主题词的位置(Loc)关系,分类效果并不理想。提出一种基于BERT与LDA的Loc-注意力(Attention)的双向长短期记忆(Bi-LSTM)模型的文本情感分析方法——BL-LABL方法。使用LDA主题模型获得每个评论的主题及其词分布,将筛选出的主题词和原文本拼接输入到BERT模型,进行词向量训练,得到包含主题信息的文本词向量以及包含文本信息的主题词向量;利用Bi-LSTM网络,加入文本的位置权重,结合注意力权重最终得到的文本特征表示为两者的加权求和;最后,再利用SoftMax分类器获得文本的情感类别。通过在两种数据集上的实验表明,该模型与传统的注意力情感分类模型相比,有效地提高了分类性能。

Traditional sentiment analysis methods do not pay attention to the positional relationship of the text relative to the topic words,so the classification effect is not ideal.In order to improve the accuracy of classification,a text sentiment analysis method based on bidirectional encoder representations from transformers(BERT)and latent Dirichlet allocation(LDA)Loc-Attention bidirectional long short-term memory(Bi-LSTM),BL-LABL,model is proposed.The LDA topic model is used to obtain the topic of each review and its word distribution,the selected topic words and original text is spliced and input into the BERT model,and word vector training is performed to obtain text word vectors containing topic information and topic word vectors containing text information.And then,Bi-LSTM is used,the location weight of the text is added and the attention weight is combined,the final text feature is expressed as the weighted sum of the two.Finally,the sentiment category of the text is obtained by the SoftMax classifier.Experiments on two kinds of datasets show that compared with the traditional attention sentiment classification model,this model effectively improves the classification performance.