基于BERT-LSTM模型的航天文本分类研究

Research on Aerospace Text Classification Based on BERT-LSTM Model

由于现有其他模型存在无法提取文本重点部分权重等问题,导致了模型分类不准确,难以适应航天文本分类工作中繁重的工作环境。因此,在融合BERT预训练模型和LSTM神经网络模型基础上,结合多特征嵌入和多网络融合方法构建BERT-LSTM模型,使用BERT模型将输入的文本转换为词向量,然后将文本序列的词向量拼接成矩阵,之后采用不同尺寸的卷积核进行卷积操作,将得到的最大特征组成特征向量集合,再输入到Bi-LSTM层进行序列建模,并采用自注意力来捕捉全局信息中的关键信息,进一步提高关键特征在文本分类中的权重。将其与TextCNN、TextRNN、DPCNN等模型进行航天文本分类任务对比试验,结果表明:本文提出的基于双向长短时记忆网络融合注意力机制模型在航天文本分类任务上相比其他模型分别提升了25.3%、25.8%和18.4%的准确率。

Due to the problems of other existing models,such as the inability to extract the weights of the key parts of the text,the model classification is inaccurate,and it is difficult to adapt to the heavy work environment in the space text classification work.Therefore,based on the fusion of the BERT pre-training model and the LSTM neural network model,we combine the multi-feature embedding and multi-network fusion methods to construct the BERT-LSTM model,using the BERT model to convert the input text into word vectors.Then,the word vectors of the text sequence are concatenated into a matrix,and different sizes of convolution kernels are used for convolution operations.The obtained maximum features are combined into a feature vector set,which is then input into the Bi-LSTM layer for sequence modeling.Self attention is used to capture key information in the global information,further improving the weight of key features in text classification.Comparison tests are conducted with TextCNN,TextRNN,DPCNN and other models for aerospace text categorization task,and the results show that the proposed model based on bi-directional long and short-term memory networks fused with the attention mechanism improves the accuracy by 25.3%,25.8%,and 18.4%compared with the other models on aerospace text categorization task,respectively.