融入语言模型和注意力机制的临床电子病历命名实体识别

Clinical Electronic Medical Record Named Entity Recognition Incorporating Language Model

临床电子病历命名实体识别(Clinical Named Entity Recognition,CNER)的主要任务是对给定的一组电子病历文档进行识别并抽取出与医学临床相关的命名实体,然后将它们归类到预先定义好的类别中,如疾病、症状、检查等实体。命名实体识别任务通常被看作一个序列标注问题。目前,深度学习方法已经被广泛应用于该任务并取得了非常好的效果。但其中大部分方法未能有效利用大量的未标注数据;并且目前使用的特征相对简单,未能深入捕捉病历文本自身的特征。针对这两个问题,文中提出一种融入语言模型和注意力机制的深度学习方法。该方法首先从未标注的临床医疗数据中训练字符向量和语言模型,然后利用标注数据来训练标注模型。具体地,将句子的向量表示送入一个双向门控循环网络(Bidirectional Gated Recurrent Units,BiGRU)和预训练好的语言模型,并将两部分的输出进行拼接。之后,将前一层的拼接向量输入另一个BiGRU和多头注意力(Multi-head Attention)模块。最后,将BiGRU和多头注意力模块的输出进行拼接并输入条件随机场(Conditional Randoin Field,CRF),预测全局最优的标签序列。通过利用语言模型特征和多头注意力机制,该方法在CCKS-2017 Shared Task2标准数据集上取得了良好的结果(F1值为91.34%)。

Clinical Named Entity Recognition(CNER)aims to identify and classify named entity such as diseases,symptoms,exams,etc.in electronic health records,which is a fundamental and crucial task for clinical and translational research.The task is regarded as a sequence labeling problem.In recent years,deep neural network methods achieve significant success in named entity recognition.However,most of these algorithms do not take full advantages of the large amount of unlabeled data,and ignore the further features from the text.This paper proposed a model which combines language model and multi-head attention.First,chara-cter embeddings and a language model are trained from unlabeled clinical texts.Then,the labeling model are trained from labeled clinical texts.In specific use,the vector representation of the sentence is sent to a BiGRU and a pre-trained language model.This paper further concatenate the output of BiGRU and the features of language model.Afterwards,the outputs are fed to another BiGRU and multi-head attention module.Finally,a CRF layer is employed to predict the label sequence.Experimental results show that the proposed method which takes advantages of language model from the text and multi-head attention mechanism gets 91.34%of F1-score on CCKS-2017 Task2 benchmark dataset.