基于K-BERT和残差循环单元的中文情感分析

Chinese Sentiment Analysis Based on K-BERT and Residual Recurrent Units

【目的】利用自然语言处理技术可以为网络舆论安全提供技术支持。为解决文本情感分析中存在的循环神经网络无法获取深层加浅层的特征信息,以及动态词向量偏离核心语义的问题,本文提出了基于K-BERT和残差循环单元的K-BERT-BiRESRU-ATT的情感分析模型。【方法】首先使用K-BERT模型获取包含背景知识的语义特征向量;之后使用提出的双向残差简单循环单元(Bidirectional Residual Simple Recurrent Unit,BiRESRU),对上下文特征进行序列提取,获取深层和浅层的特征信息;然后利用注意力机制对BiRESRU的输出进行关键词权重增强;最后使用softmax进行结果分类。【结果】在ChnSentiCorp和weibo数据集上,分别达到了95.6%和98.25%的准确率;在计算速度上较使用其他循环网络每轮迭代减少了接近5分钟,提高了计算效率。【结论】K-BERT-BiRESRU-ATT解决了动态词向量偏离核心语义的问题,获得了深层加浅层的特征信息,加速模型计算的同时也提高了分类准确率,但仍对计算能力有较大需求。

[Objective]The use of natural language processing technology can provide technical support for the security of network public opinion.In order to solve the problem that the recurrent neural network in text sentiment analysis cannot obtain the feature information of deep and shallow layers,and the dynamic word vector deviates from the core semantics,a K-BERT-BiRESRU-ATT based on K-BERT and the residual recurrent unit is proposed.[Methods]First,the K-BERT model is used to obtain the semantic feature vector containing background knowledge;Then,the proposed Bidirectional Residual Simple Recurrent Unit(BiRESRU)is used to extract the sequence of the contextual features to obtain deep and shallow feature information;After that,the attention mechanism is used to enhance the keyword weight of the output of BiRESRU;Finally softmax is used to classify the results.[Results]On the ChnSentiCorp and Weibo datasets,the accuracy rates were 95.6%and 98.25%,respectively;the calculation time was reduced by nearly 5 minutes per iteration compared with other recurrent networks,and the computational efficiency was improved.[Conclusions]K-BERT-BiRESRU-ATT solves the problem of the dynamic word vector deviation from the core semantics,obtains the feature information of deep and shallow layers,accelerates the model calculation,and improves the classification accuracy.But it still has a large demand for computing ability.