国防科技领域两阶段开放信息抽取方法

Two-stage open information extraction method for the defence technology field

互联网开源渠道蕴含大量国防科技信息资源,是获取高价值军事情报的重要数据来源。国防科技领域开放信息抽取(open information extraction,OpenIE)旨在从海量信息资源中进行主谓宾-宾补(SAO-C)结构元组抽取,其对于国防科技领域本体归纳、知识图谱构建等具有重要意义。然而,相比其他领域的信息抽取,国防科技领域开放信息抽取面临元组重叠嵌套、实体跨度长且难识别、领域标注数据缺乏等问题。本文提出一种国防科技领域两阶段开放信息抽取方法,首先利用基于预训练语言模型的序列标注算法抽取谓语,然后引入多头注意力机制来学习预测要素边界。结合领域专家知识,利用基于实体边界的标注策略构建了国防科技领域标注数据集,并在该数据集上进行了实验,结果显示该方法的F1值在两阶段上比长短期记忆结合条件随机场(LSTM+CRF)方法分别提高了3.92%和16.67百分点。

[Objective]The abundant information resources available on the internet about defense technology are of vital importance as data sources for obtaining high-value military intelligence.The aim of open information extraction in the field of defense technology is to extract structured triplets containing subject,predicate,object,and other arguments from the massive amount of information available on the internet.This technology has important implications for ontology induction and the construction of knowledge graphs in the defense technology domain.However,while information extraction experiments in the general domain yield good results,open information extraction in the defense technology domain faces several challenges,such as a lack of domain annotated data,arguments overlapping unadaptability,and unrecognizable long entities.[Methods]In this paper,an annotation strategy is proposed based on the entity boundaries,and an annotated dataset in the defense technology field combined with the experience of domain experts was constructed.Furthermore,a two-stage open information extraction method is proposed in the defense technology field that utilizes a pretrained language model-based sequence labeling algorithm to extract predicates and a multihead attention mechanism to learn the prediction of argument boundaries.In the first stage,the input sentence was converted into an input sequence<[CLS],input sentence[SEP]>,and the input sequence was encoded using apretrained language model to obtain an implicit state representation of the input sequence.Based on this sentence representation,a conditional random field(CRF)layer was used to predict the position of the predicates,i.e.,to predict the BIO labels of the words.In the second stage,the predicated predicates from the first stage were concatenated with the original sentence and converted into an input sequence<[CLS],predicate[SEP],and input sentence[SEP]>,which was encoded using apretrained language model to obtain an implicit state representation of the input sequence.This representation was then fed to a multihead pointer network to predict the position of the argument.The predicted position was tagged with the actual position to calculate the cross-entropy loss function.Finally,the predicates and the arguments predicted by the predicate and argument extraction models were combined to obtain the complete triplet.[Results]The experimental results from the extensive experiments conducted on a self-built annotated dataset in the defense technology field reveal the following.(1)In predicate extraction,our method achieved a 3.92%performance improvement in the F1value as compared to LSTM methods and more than 10%performance improvement as compared to syntactic analysis methods.(2)In argument extraction,our method achieved a considerable performance improvement of more than 16%in the F1value as compared to LSTM methods and about 11%in the F1value as compared to the BERT+CRF method.[Conclusions]The proposed two-stage open information extraction method can overcome the challenge of arguments overlapping unadaptability and the difficulty of long-span entity extraction,thus improving the shortcomings of existing open information extraction methods.Extensive experimental analysis conducted on the self-built annotated dataset proved the effectiveness of the proposed method.