基于BERT模型的航天科技开源情报分类

Classification of open source intelligence of aerospace science and technology based on BERT Model

提出一种基于BERT模型的航天科技开源情报分类算法。通过双向Transformer机制捕捉航天科技开源情报中句子间的关系;采用Multi-head Self-attention机制关注文本中的大量专有名词;结合Softmax分类器对提取的特征进行分类。与TextRCNN、DPCNN等主流语言模型相比,该算法在测试集上的准确率分别提升1.7%、3.33%,验证了该算法在航天科技开源情报分类上的有效性。

A classification algorithm of open source intelligence for aerospace science and technology is proposed based on the BERT model.Using the two-way Transformer mechanism,the relationship between sentences in the aerospace science and technology open source intelligence is captured;the Multi-head Self-attention mechanism is adopted to focus on a large number of proper nouns in the text;the Softmax classifier is combined to classify the extracted features.Compared with mainstream language models such as TextRCNN and DPCNN,the accuracy of the algorithm on the test set is increased by 1.7%and 3.33%respectively,which verifies the effectiveness of the algorithm in the classification of aerospace science and technology open source intelligence.