CGGA:一种CNN与并行门控机制混合的文本分类模型

CGGA:Text Classification Model Based on CNN and Parallel Gating Mechanism

针对中文文本分类准确率低、单一的卷积神经网络模型难以提取多方面特征的问题,本文提出一种基于CNN的并行门控机制的混合文本分类模型——CGGA(Convolutional Neural Network with parallel gating unit and attention mechanism).利用卷积提取文本的局部特征,并加入双向门控循环单元对数据进行上下文数据建模,提取关系特征,同时,引入门控Tanh-ReLU单元进行进一步的特征筛选,从而控制信息向下层流动的力度,并且减轻梯度弥散,提高模型分类准确率.最后,使用多头注意力机制进行权重更新计算,以提高在相应文本类别上的输出,进而优化模型分类性能.实验结果显示,本文提出的文本分类模型和分类算法,在THUCNews数据集和搜狐数据集上,比基线模型的宏平均精确率分别提高了2.24%、6.78%.

For Chinese text classification accuracy rate is low,the single Convolutional Neural Networks model is difficult to extract multiple features,this paper proposes a hybrid text classification model based on parallel gating mechanism of CNN,named CGGA(Convolutional Neural Network with parallel gating unit and attention mechanism).Convolution is used to extract local features of the text,and the Bidirectional Gated Recurrent Unit is added to conduct context data modeling to extract relational features,at the same time,the Gated Tanh-ReLU Units was introduced for further feature screening, so as to control the dynamics of information flow to the lower layer,reduce the gradient dispersion and improve the classification accuracy of the model.Finally,the Multi-Head Aattention mechanism is used to calculate the weight update to improve the output of the corresponding text categories,so as to optimize the classification performance of the model.Experimental results show that the text classification model and classification algorithm proposed in this paper have improved the macro average precision of the THUCNews dataset and sohu dataset by 2.24% and 6.78%, respectively,compared with the baseline model.