结合BERT和卷积双向简单循环网络的文本情感分析

TEXT EMOTION ANALYSIS BASED ON BERT AND CNN-BISRU

针对传统预训练模型无法解决多义词表征问题和抽取的情感特征能力不足等问题,提出一种结合BERT和卷积双向简单循环网络的文本情感分类模型。用BERT预训练得到融合句子语境的动态词向量;用多粒度卷积神经网络对词向量特征进行二次抽取,池化后注入局部响应归一化层LRN来归一化特征图,以生成句子整体特征表示;利用双向简单循环单元进一步学习上下文语义信息;使用Softmax回归得出文本的情感倾向。实验结果表明,结合BERT和卷积双向简单循环网络的文本情感分类模型能获得更高的准确率,注入的LRN层和并行化循环网络有效提高模型性能,具有较好的实用价值。

Aimed at the problem that the traditional language model can not solve the problem of word ambiguity in word vector representation and not be able to capture more semantic features in existing text sentiment analysis methods, the text emotion analysis based on BERT and CNN-BiSRU model is proposed. The BERT was used to obtain the word dynamic vector representation that integrated text semantics. The word vector features was secondly extracted by multi-kernel convolution neural network(CNN), and the local response normalization layer(LRN) was injected to normalize the feature map after pooling to generate the overall feature representation of the sentence. Bidirectional simple recurrent unit(BiSRU) was used to further extract semantic context information. Softmax function was utilized to determine the emotional tendency of the text. The experimental results show that the model based on BERT and CNN-BiSRU can achieve higher accuracy, the injected LRN layer and the parallel circular network can improve the performance of this model. It is of great value in practical application.