基于RoBERTa和多头注意力的简历命名实体识别方法

Resume Name Entity Recognition Method Based on RoBERTa and Multi-head Attention

针对传统简历实体识别存在一词多义和训练时间长的问题,提出了一种新的简历命名实体识别模型。通过RoBERTa预训练模型获取具有上下文关系的字向量,结合BiGRU和多头注意力机制(Multi-head Attention,MHA)层提取全局信息和局部相关性信息,采用CRF层修正解码确定最终标签,同时裁剪RoBERTa预训练模型。实验表明,该模型在中文电子简历数据集取得95.97%的F 1值,高于其他主流模型,且相较于未剪枝的模型提升0.43%,减少1/5训练时间。

A new named entity recognition model for resumes was proposed to address the problems of multiple word meanings and long training time in traditional resume entity recognition.The word vectors with contextual relationships were obtained by RoBERTa pre-training model,the global information and local relevance information were extracted by combining BiGRU and Multi-head Attention(MHA)layers.The final labels were determined by corrective decoding using CRF layer,and the RoBERTa pre-training model was cropped at the same time.The experiment shows that the F 1 value of the new model in Chinese electronic resume dataset is 95.97%,which is higher than other mainstream models.Compared with the non pruning model,the F 1 value of the new model is increased by 0.43%and the training time is reduced by 1/5.