基于ALBERT、Bi-GRU和注意力机制的文本情感分析

Text Sentiment Analysis Based on ALBERT,Bi-GRU and Attention Mechanism

针对静态词向量方法不能很好地解决一词多义,长短时记忆网络参数量较多、训练时间过长等不足,提出将ALBERT预训练模型、双向门控循环单元、多头注意力机制融合在一起,构建了一个微博文本情感预测模型.首先,通过ALBERT模型获取文本动态词向量;然后采用双向门控循环单元提取文本特征;接着引入多层注意力机制捕获文本序列中的重要信息;最后,通过Softmax进行情感分类.实验结果表明:所提出的模型与传统模型相比,能有效提取文本的特征,与静态词向量相比,模型准确率提升1.76%,与长短时记忆网络相比,参数数量下降25%,训练效率提升20%,有较好的实用价值.

In view of that the static word vector method can not solve the problem of polysemy,Long Short-Term Memory Network has many parameters,long training time.In this paper,a Microblog senti‐ment prediction model is constructed by integrating the BERT pre training model,Bi-GRU and multi head attention mechanism.Firstly,the text dynamic word vector is obtained by Bert model;Secondly,the text feature is extracted by using the Bi-GRU;Thirdly,the multi-layer attention mechanism is introduced to capture the important information in the text sequence;Finally,the emotions are classified by Softmax.The experimental results show that the proposed model can effectively extract text features compared with the traditional model.Compared with the static word vector,the accuracy of the model is improved by 1.76%.Compared with Bi LSTM,the number of parameters is reduced by 25%,and the training efficiency is improved by 20%.So It has great practical value.