基于BERT嵌入的中文命名实体识别方法

Chinese Named Entity Recognition Method Based on BERT Embedding

在基于神经网络的中文命名实体识别过程中,字的向量化表示是重要步骤,而传统的词向量表示方法只是将字映射为单一向量,无法表征字的多义性.针对该问题,通过嵌入BERT预训练语言模型,构建BERT-BiGRU-CRF模型用于表征语句特征.利用具有双向Transformer结构的BERT预训练语言模型增强字的语义表示,根据其上下文动态生成语义向量.在此基础上,将字向量序列输入BiGRU-CRF模型中进行训练,包括训练整个模型和固定BERT只训练BiGRU-CRF2种方式.在MSRA语料上的实验结果表明,该模型2种训练方式的F1值分别达到95.43%和94.18%,优于BiGRU-CRF、Radical-BiLSTM-CRF和Lattice-LSTM-CRF模型.

In Chinese Named Entity Recognition(NER)based on neural network,the vectorized representation of words is an important step.Traditional representation methods for word vectors only map a word to a single vector,and cannot represent the polysemy of a word.To address the problem,this paper introduces the BERT pretrained language model to build a BERT-BiGRU-CRF model for representation of sentence characteristics.The BERT pretrained language model with bidirectional Transformer structure is used to enhance the semantic representation of words and generate semantic vectors dynamically based on their context.On this basis,the word vector sequence is input into the BIGR-CRF model to train the whole model,or train the BIGR-CRF part only with BERT fixed.Experimental results on MSRA data show that the F1 value in the two training modes of this proposed model reaches 95.43%and 94.18%respectively,which is better than that of the BIGRU-CRF,the RADICAL-BILSTM-CRF and the GRAIN-LSTM-CRF models.