结合自注意力机制的神经网络文本分类算法研究

NEURAL NETWORK TEXT CLASSIFICATION ALGORITHM COMBINING SELF-ATTENTION MECHANISM

在长文本数据中存在很多与主题不相关词汇,导致这些文本数据具有信息容量大、特征表征不突出等特点。增加这些文本中关键词汇的特征影响,是提高文本分类器性能需要解决的问题。提出一种结合自注意力机制的循环卷积神经网络文本分类模型RCNN_A。注意力机制对文本词向量计算其对正确分类类别的贡献度,得到注意力矩阵,将注意力矩阵和词向量矩阵相结合作为后续结构的输入。实验结果表明,RCNN_A在10类搜狗新闻数据集上,得到了97.35%的分类正确率,比Bi-LSTM(94.75%)、Bi-GRU(94.25%)、TextCNN(93.31%)、RCNN(95.75%)具有更好的文本分类表现。通过在深度神经网络模型中引入注意力机制,能够有效提升文本分类器性能。

There are many words irrelevant to the subject in the long text data,which leads to the characteristics of large information capacity and weak outstanding feature representation.To improve the performance of text classifier,it is necessary to increase the characteristic influence of key words'features.This paper proposes a circular convolutional neural network text classification model RCNN_A combining self-attention mechanism.The attention mechanism calculated the contribution of the text words vector to the correct classification category,obtained the attention matrix,and combined the attention matrix and the word vector matrix as the input of the subsequent structure.Experimental results show that,RCNN_A obtains a classification accuracy of 97.35%on the 10 categories of Sogou datasets,and hasbetter text classification performance than Bi-LSTM(94.75%),Bi-GRU(94.25%),TextCNN(93.31%),and RCNN(95.75%).The text classifier performance can be effectively improved by introducing self-attention mechanism in the deep neural network model.