基于异质图神经网络的知识增强文本理解研究

Research on Knowledge Enhanced Text Understanding Based on Heterogeneous Graph Neural Network

针对当下流行的语言模型如BERT,即使其在大量文本下预训练而具有了较好的文本理解能力,但由于文本中知识共现的稀疏性导致模型难以理解复杂语义的场景的问题,提出一种引入知识图谱结合异质图网络增强文本理解的方法。该方法根据文本中的实体构造异质图,并基于边采样机制生成子视图。引入R-GCN聚合异质子视图下的邻居信息,并约束同节点在不同视图下基于相似性的度量损失。最终得到知识增强的文本表示。在公开数据集WinoGrande上的结果表明,该方法在准确率指标下有着明显的提升,相比于未增强时提升了1.3%。

Nowadays,popular language models such as BERT have good reading comprehension ability because of the pre-training of large-scale corpus.However,due to the sparseness of knowledge co-occurrence in the sentences,it is difficult for the model to understand scenes with complex semantics.To solve this problem,a method of introducing knowledge graph combined with heterogeneous graphs to enhance reading comprehension is proposed.This method constructs heterogeneous graphs based on the entities in the sentence,and generates sub-views based on the edge-dropout.The R-GCN aggregates the neighbor information in the heterogeneous sub-views,and constrains the similarity-based metric loss of the same node in different views.Finally,the knowledge-enhanced textual representation is obtained.The results on the WinoGrande show that this method has a significant improvement in accuracy,which is 1.3%higher than without knowledge enhancement.