基于词嵌入和BiLCNN-Attention混合模型的政务文本分类方法

Method of government text classification based on word embedding and BiLCNN-Attention hybrid model

针对政务文本分析语境复杂、分类准确率低的问题,提出基于BERT词嵌入和BiLCNN-Attention混合模型的文本分类方法。首先采用BERT模型对政务文本进行词嵌入向量表示,然后混合使用双向长短时记忆网络BiLSTM和卷积神经网络CNN,同时引入注意力机制进行特征提取,融合了时序特征及局部特征并使特征得到强化,最后使用Softmax进行文本分类。实验表明,BERT词嵌入处理后混合模型的准确率较CNN和BiLSTM模型分别提升了3.9%和2.51%。

Aiming at the problems of complex context and low classification accuracy of government texts,a text classification method based on BERT word embedding and BiLCNN-Attention hybrid model is proposed.Firstly,the BERT model is used to represent the word embedding vector of the government text.Then,BiLSTM and CNN are mixed,and the attention mechanism is introduced for feature extraction.The timing features and local features are integrated and strengthened.Finally,Softmax is used for text classification.Experiments show that the accuracy of the hybrid model after BERT word embedding is improved by 3.9%and 2.51%compared with CNN and BiLSTM models,respectively.