基于BERT的电力领域无监督分词方法

Unsupervised word segmentation method in power domain based on BERT

目前,已有一些分词工具实现了通用领域分词,而在电力领域中进行分词面临相关文本少,缺乏已标注数据且人工标注工作成本高等问题。为了克服这些困难,提出了一种基于BERT特征编码的无监督分词工具,采用遮蔽语言模型(MLM),基于BERT计算部分被遮蔽的句子的特征编码来度量句子各部分相似度,并将相似度较低的部分进行拆分,再通过N-Gram对于拆分结果进行重新组合,实现电力领域的无监督分词。实验结果表明,文中方法在通用领域优于现有分词工具,尤其在电力领域的分词任务中取得了较好的效果。

At present,some word segmentation tools have realized the word segmentation in general domain,however,problems such as few related texts,missing labeled data,and high cost of manual labeling are existed in power domain.To overcome these difficulties,this paper puts forward an unsupervised word segmentation tool based on BERT.Masked Language Model(MLM) is adopted.Besides,on the basis of the feature codes of sentences partially masked by BERT's calculation,the similarity of each part of the sentence are measured,and the parts with low similarity would be split up.Then N-Gram combines the results which are over-segmentation to realize the unsupervised word segmentation in power domain.The experiment results show that the proposed method is superior to the existing word segmentation tools in general fields,especially in power domain.