密集池化连接和短语注意力下的文本分类算法

Text Classification Based on Dense-Pooling Connection and Phrase Attention

为了解决在文本分类中神经网络训练时产生的梯度消失、特征信息丢失以及注意力机制短语维度组合不匹配的问题,提出一种基于密集池化连接和短语注意力机制的文本分类算法。首先,通过密集池化连接中的残差网络部分进行特征提取,可有效缓解梯度消失问题;其次,通过池化层复用重要特征,改善特征信息丢失问题;最后,通过改进常规注意力机制,提出短语注意力机制,可灵活得到不同阶短语之间的联系,解决常规注意力机制短语维度不匹配问题。结果表明,该模型在对比模型中取得了最好的效果,在相同的新闻数据集中准确率可达92.7%,同时还对3个对比模型的收敛性和分类准确性进行分析,可见改进后的模型可以有效缓解梯度消失,并且解决短语维度组合不匹配问题,从而提高了分类准确性。

In order to solve problems for the disappearance of gradients,the deficiency of text feature and the mismatch of extracting phrase features in attention mechanism during the training of neural network in text classification,a new method base on dense-pool connection and phrase attention mechanism was proposed.Firstly,the method was used to extract features while alleviating the gradient disappearance problem through the residual network and reuse important features through dense pooling connection.Secondly,the phrase attention mechanism was used to solve the problem of phrase dimension mismatch in the traditional attention mechanism.Finally,the results show that the accuracy of the model can achieve 92.7%in the AG news dataset for all variants.In addition,the convergence and classification accuracy of three comparison models were analyzed in different hyperparameters.It is concluded that the improved model can effectively alleviate the disappearance of gradients and solve the problem of phrase feature extraction,thereby the classification accuracy is improved.