用于文本分类的CNN_BiLSTM_Attention混合模型

CNN_BiLSTM_Attention Hybrid Model for Text Classification

文本分类是许多自然语言处理任务的基础。卷积神经网络可以提取文本的短语级特征,但是不能很好地捕获文本的结构信息;循环神经网络可以提取文本的全局结构信息,但是对关键模式信息捕获能力不足;而注意力机制能够学习到不同词或短语对文本整体语义的分布,关键的词或短语会被分配较高的权重,但是同样对全局结构信息不敏感。另外,现有模型大多只考虑词级信息,而忽略了短语级信息。针对上述模型中存在的问题,文中提出一种融合CNN,RNN,Attention的混合模型,该模型同时考虑不同层次的关键模式信息和全局结构信息,并把它们融合起来得到最终的文本表示,最后把文本表示输入softmax层进行分类。在多个文本分类数据集上进行了实验,实验结果表明该模型相较于现有模型可以实现更高的准确率。此外,还通过实验分析了模型的不同组件对模型性能的影响。

Text classification is the basis of many natural language processing tasks.Convolutional neural network(CNN)can be used to extract the phrase level features of text,but it can’t capture the structure information of text well;Recurrent neural network(RNN)can extract the global structure information of text,but its ability to capture the key pattern information is insufficient.Attention mechanism can learn the distribution of different words or phrases to the overall semantics of text,key words or phrases will be assigned higher weights,but it is not sensitive to global structure information.In addition,most of the existing models only consider word level information,but ignore phrase level information.In view of the problems in the above models,this paper proposes a hybrid model which integrates CNN,RNN and attention.The model considers the key pattern information and global structure information of different levels at the same time,and fuses them to get the final text representation.Finally,the text representation is input to the softmax layer for classification.Experiments on multiple text classification datasets show that the model can achieve higher accuracy than the existing models.In addition,the effects of different components on the performance of the model are analyzed through experiments.