融合BERT-WWM与注意力机制的电力设备缺陷实体识别研究

An Examination of the Integration of BERT-WWM and Attention Mechanism in Entity Recognition of Electrical Equipment Defects

针对电力设备运行和维护中所产生的大量碎片化、非系统性以及相关性不足设备缺陷文本,提出了一种电力设备缺陷文本识别模型。使用基于全词掩码的预训练模型(bidirectional encoder representation from transform-ers,BERT)替换基于随机掩码的BERT模型,提高了对电力词汇的理解力。使用双向长短期记忆网络(bidirection-al long short-term memory,BiLSTM)提高了模型捕获上下文信息的能力,并提高了模型的鲁棒性,引入注意力机制(attention)可以更好地捕获电力设备缺陷实体之间的复杂依赖关系,从而进一步提升模型的表现。实验结果显示,该模型准确率、召回率、F1值分别为96.26%、96.94%、96.60%,在地点、缺陷内容和设备三种实体上的F1值均优于其他模型。

Addressing the pervasive issue of disparate,non-systematic,and inadequately correlated defect text generated during the operation and maintenance of electrical equipment,a novel model for the recognition of electrical equipment defect text is put forth.The bidirectional encoder representation from transformers(BERT)model predicated on whole word masking is utilized,supplanting the traditional BERT model pre-mised on random masking,which augments the comprehension of electrical lexicon.The integration of bidirec-tional long short-term memory(BiLSTM)into the model fortifies the capacity to apprehend contextual informa-tion,bolstering the model’s robustness.Moreover,the incorporation of the Attention mechanism enables the model to capture sophisticated dependencies between entities of electrical equipment defects,thereby further en-hancing the model’s performance.Empirical results corroborate that the accuracy,recall,and F1 score of the model are an impressive 96.26%,96.94%and 96.60%respectively.Furthermore,the F1 scores for the loca-tion,defect content,and equipment entities all surpass those of competing models,underscoring the superiority and efficacy of the proposed model.