基于链接实体回放的多源知识图谱终身表示学习

Lifelong Representation Learning of Multi-sourced Knowledge Graphs via Linked Entity Replay

知识图谱存储大量的结构化知识和丰富的语义信息,已被广泛应用于知识驱动的智能软件.随着智能应用的不断发展,它们对知识图谱的需求也在发生变化.而单一知识图谱往往具有数据不完备等缺点,难以满足需求.因此,支持新数据来源、融合多源知识已成为迫切需求.传统的知识图谱表示学习和应用范式只考虑单一图谱,忽视了不同图谱间的知识迁移.多源知识图谱联合训练虽然可以带来性能提升,但不支持新增知识图谱的拓展表示学习.鉴于此,提出了多源知识图谱终身表示学习的新范式.给定一个知识图谱序列,终身表示学习的目标是在学习新知识图谱的同时,从已学习的知识图谱与模型中获得知识迁移.为实现这一目标,提出了一个基于链接实体回放的多源知识图谱终身表示学习框架.首先,设计了一个以Transformer为编码器的知识图谱表示学习模型作为框架核心,利用关系相关性进行实体的链接预测;其次,提出了链接子图构造方法,基于实体对齐构建并回放新增知识图谱和已有知识图谱之间的链接子图进行终身学习和知识迁移;最后,采用动态结构方法,为每个知识图谱存储相应的模型参数快照来避免灾难性遗忘.多个链接预测基准数据集上的实验结果表明:所提出的表示学习模型可以取得最先进的性能,且提出的终身表示学习框架可以实现有效的知识迁移.

Knowledge graphs(KGs)store a great amount of structured knowledge and semantic information.They have been widely used by many knowledge-powered intelligent applications.With the rapid development of these applications,their requirements for knowledge also change.A single KG usually suffers from the incompleteness issue and is therefore unable to meet the requirement.This suggests an urgent demand for supporting new data sources and fusing multi-sourced knowledge.The conventional paradigm for KG representation learning and application only considers a single KG while ignores the knowledge transfer between different sources.Joint representation learning on multi-sourced KGs can bring performance improvement,but it cannot support the extended representation learning of new KGs.To resolve these issues,this paper presents a new paradigm,i.e.,lifelong representation learning on multi-sourced KGs.Given a sequence of multi-sourced KGs,lifelong representation learning aims at benefiting from the previously-learned KG and embedding model when learning a new KG.To this end,this study proposes a lifelong learning framework based on linked entity replay.First,it designs a Transformer-based KG embedding model that leverages relation correlations for link prediction between entities.Second,it proposes a linked subgraph generation method.It leverages the entity alignment between different sources to build the subgraph and replays the linked entities to enable lifelong learning and knowledge transfer.Finally,it uses a dynamic model structure with model parameters and embeddings stored for each KG to avoid catastrophic forgetting.Experiments on benchmarks show that the proposed KG embedding model can achieve the state-of-the-art performance in link prediction,and the lifelong representation learning framework is effective and efficient in multi-sourced knowledge transfer compared with baselines.