知识增强的BERT短文本分类算法

Classification algorithm for short text based on knowledge enhanced BERT

为解决短文本信息不全且缺乏领域知识导致关键信息难以充分挖掘而造成的深度学习模型分类性能不足等问题,提出一种知识增强的双向编码器表示转换器(BERT)短文本分类算法(KE-BERT)。提出一种建模短文本与领域知识的方法,通过知识图谱进行领域知识的引入;提出一种知识适配器,通过知识适配器在BERT的各个编码层之间进行知识增强。通过在公开的短文本数据集上,将KE-BERT与其它深度学习模型相比较,该模型的F1均值和准确率均值达到93.46%和91.26%,结果表明了所提模型性能表现良好。

To solve the problem of poor classification performance of deep learning models caused by incomplete short text information and lack of domain knowledge,a knowledge enhanced bidirectional encoder representation from transformers(BERT)short text classification algorithm(KE-BERT)was proposed.A method of modeling short text and domain knowledge was proposed,and the domain knowledge was introduced by knowledge graph.A knowledge adapter was proposed to enhance knowledge between the encoding layers of BERT.By comparing KE-BERT with other deep learning models on the published short text dataset,the F1 mean and accuracy mean of this model reach 93.46%and 91.26%,indicating that the proposed model has good performance.