基于特征融合的糖尿病命名实体识别

Diabetes named entity recognition based on feature fusion

针对医学糖尿病领域命名实体识别中存在实体种类多样性、数据稀缺等问题,提出了基于特征融合的糖尿病命名实体识别方法。以BERT+BILSTM+CRF为基准模型,在3方面进行改进。首先,使用预训练模型RoBERTa-wwm-ext作为模型嵌入层,提供字符级嵌入,利用其在训练阶段进行全词掩码来获取含有先验知识的语义表示。其次,使用双向长短时记忆网络和迭代膨胀卷积神经网络并行提取特征,以获取不同粒度的特征。同时,结合注意力机制进行动态特征融合,从而更好地理解数据的关键信息,以获得更丰富的上下文特征。最后,采用条件随机场进行解码,获得最终的预测结果。该模型在包含18种实体类别的中文糖尿病数据集DiaKG上的F1值达到了79.58%,实验结果表明,与High-Order MKGragh模型相比,该模型的F1值提升了5.38%,充分说明了特征融合的方法能够有效识别糖尿病实体。

To overcome the challenges of entity diversity and data scarcity in diabetes named entity recognition,feature fusion based named entity recognition is proposed.With BERT+BILSTM+CRF as the benchmark model,improvements are made in 3 aspects.(1)The pre-trained model RoBERTa-wwm-ext is introduced as the model embedding layer to provide character-level embedding,and the whole word mask is used in the training stage to obtain semantic representation containing prior knowledge.(2)bidirectional long short-term memory network and iterated dilated convolutional neural network are used to extract features in parallel to obtain features of different granularities.At the same time,the dynamic feature fusion is combined with the attention mechanism to better understand the key information of the data,thus obtaining richer contextual features.(3)The conditional random field is decoded to obtain the final prediction results.The proposed model achieves an F1 value of 79.58%which is 5.38%higher than High-Order MKGraph on DiaKG,a Chinese diabetes data set containing 18 entity categories,fully demonstrating that the feature fusion based method can effectively identify diabetic entities.