基于BERT的多源知识库索引对齐算法

Multi⁃source knowledge base index alignment based on BERT

各类百科知识库是当代网民学习生活不可或缺的知识来源,但是单个知识库存在实体覆盖面低、实体信息缺失等问题;而不同知识资源和知识社区之间资源描述结构存在较大差异,阻碍了数据的共享、集成与复用。针对规模巨大的百科知识库,提出一种基于Bert的多源知识库索引算法,利用Bert预训练语言模型对实体的非结构化文本建模,构建特征向量计算实体相似度,对异构知识库具有良好的通用性。在此基础上,针对实体指称与实体上下文的关键词提出两种索引构建方式,有效提高了实体对齐的效率和准确率。通过5组测试数据的验证,该算法的准确率能够达到91%~96.22%,同时召回率高达92.23%~97.33%。与此同时,该算法的索引结构在缩减率达到99􀆰94%的同时保证了98.52%的完整性。该算法可以实际应用于多源知识库的实体对齐工作。

All kinds of encyclopedic knowledge bases are indispensable knowledge sources for contemporary netizens to study and live,but the problems of single knowledge base include low entity coverage and missing entity information.Moreover,the resource description structure of different knowledge resources and knowledge communities is a biger difference,hindering the sharing,integration and reuse of data.Aiming at the large scale encyclopaedic knowledge bases,an index algorithm for multi⁃source knowledge bases based on BERT is proposed by using BERT pre⁃training language model to model the unstructured text of entities and construct feature vectors to calculate entity similarity,thus it has good universality to heterogeneous knowledge bases.On the basis of the algorithm,two index construction methods are proposed for the key words of the entity reference and the entity context,improving the efficiency and the accuracy of entity alignment.Through the verification of five groups of test data,the accuracy of the algorithm can reach 91%—96.22%,and the recall rate is as high as 92.23%—97.33%.Meanwhile,the reduction ratio of index structure of the algorithm can be reduced by 99.94%,ensuring the integrity of 98.52%.The algorithm can be applied to the entity alignment tasks of the multi⁃source knowledge bases.