基于BERT的警情文本命名实体识别

Alarm text named entity recognition based on BERT

针对警情领域关键实体信息难以识别的问题,提出一种基于BERT的神经网络模型BERT-BiLSTMAttention-CRF用于识别和提取相关命名实体,且针对不同案由设计了相应的实体标记注规范。该模型使用BERT预训练词向量代替传统Skip-gram和CBOW等方式训练的静态词向量,提升了词向量的表证能力,同时解决了中文语料采用字向量训练时词语边界的划分问题;还使用注意力机制改进经典的命名实体识别(NER)模型架构BiLSTM-CRF。BERT-BiLSTM-Attention-CRF模型在测试集上的准确率达91%,较CRF++的基准模型提高7%,也高于BiLSTM-CRF模型86%的准确率,其中相关人名、损失金额、处理方式等实体的F1值均高于0.87。

Aiming at the problem that the key entity information in the police field is difficult to recognize,a neural network model based on BERT(Bidirectional Encoder Representations from Transformers),namely BERT-BiLSTMAttention-CRF,was proposed to recognize and extract related named entities,in the meantime,the corresponding entity annotation specifications were designed for different cases.In the model,the BERT pre-trained word vectors were used to replace the word vectors trained by the traditional methods such as Skip-gram and Continuous Bag of Words(CBOW),improving the representation ability of the word vector and solving the problem of word boundary division in Chinese corpus trained by the character vectors.And the attention mechanism was used to improve the architecture of classical Named Entity Recognition(NER)model BiLSTM-CRF.BERT-BiLSTM-Attention-CRF model has an accuracy of 91%on the test set,which is 7%higher than that of CRF++Baseline,and 4%higher than that of BiLSTM-CRF model.The F1 values of the entities are all higher than 0.87.