基于参数迁移的领域命名实体识别方法

Domain Named Entity Recognition Method Based on Parameter Transfer Learning

[目的/意义]命名实体识别是自然语言处理领域中的基础任务,基于深度学习的方法在通用领域的命名实体中取得了显著成果,但在特定领域识别效果不佳。为了解决工业信息化领域标注数据不足,数据特征差异较大、模型难以扩展的问题,首先提出了一种基于Transformer的有限区间命名实体识别模型。[方法/过程]采用预训练模型对文本进行分布式表示,然后利用基于有限区间的标注方法对输入序列进行标注,解决传统标注法在训练过程中可能导致的序列标注不一致的问题。在此基础上,引入迁移学习策略,采用参数共享的方式,将通用领域的命名实体识别模型迁移到工业信息化领域,并在工业信息化领域数据集上进行微调,最终获得在工业信息化领域上表现良好的模型。[结果/结论]实验结果表明,本文提出的有限区间命名实体识别模型在工业信息化领域数据集上的准确率较基线模型提高了8.7%,基于参数迁移的领域命名实体识别方法在人民日报语料和工业信息化领域数据集上的准确率和综合指标F值相较未使用迁移学习的模型分别提高了3.1%和1.1%,证明了迁移策略的有效性。

[Objective/Significance]Named entity recognition is a fundamental task in natural language processing,and deep learning-based methods have achieved remarkable results in general domains,but not in specific domains.Aiming at the problems of insufficient labeling samples,quite differences in data features and difficulty in model expansion,this paper introduces a limited span-based transformer classifier for named entity recognition model(Span-based Transformer Classifier for Named Entity Recognition,STCNER).[Methods/Process]The model takes advantage of the features extraction of Encoder in Transformer and combines with the limited span-based labeling method,which solves the problem of the sequence labeling inconsistency caused by traditional labeling method in the training process.On this basis,then introduce the transfer learning strategy which adopt the parameter sharing method to transfer the named entity recognition model in general domains to the specific domains.After fine-tuning it on the domain-specific dataset,the model performs well in specific domain.[Results/Conclusions]The experimental results show that the accuracy of STCNER model is 8.7%higher than the baseline model on the dataset in the industrial informatization field.Compared with the model without transfer learning,the accuracy and F-scores are improved by 3.1%and 1.1%respectively on the corpus of People's Daily and the data set in the industrial informatization field,which proves the effectiveness of the transfer strategy.