融合TuckER嵌入和强化学习的知识推理

Knowledge Reasoning Combining TuckER Embedding and Reinforcement Learning

知识推理是补全知识图谱的重要方法,旨在根据图谱中已有的知识,推断出未知的事实或关系.针对多数推理方法仍存在没有充分考虑实体对之间的路径信息,且推理效率偏低、可解释性差的问题,提出了将TuckER嵌入和强化学习相结合的知识推理方法TuckRL(TuckER embedding with reinforcement learning).首先,通过TuckER嵌入将实体和关系映射到低维向量空间,在知识图谱环境中采用策略引导的强化学习算法对路径推理过程进行建模,然后在路径游走进行动作选择时引入动作修剪机制减少无效动作的干扰,并将LSTM作为记忆组件保存智能体历史动作轨迹,促使智能体更准确地选择有效动作,通过与知识图谱的交互完成知识推理.在3个主流大规模数据集上进行了实验,结果表明TuckRL优于现有的大多数推理方法,说明将嵌入和强化学习相结合的方法用于知识推理的有效性.

Knowledge reasoning is an important method to complement a knowledge graph, which aims to infer unknown facts or relations according to the existing knowledge in the graph. As the path information between entity pairs is not fully considered in most reasoning methods, the reasoning shows low efficiency and poor interpretability. To solve this problem, this study proposes TuckER embedding with reinforcement learning(TuckRL), a knowledge reasoning method that combines TuckER embedding and reinforcement learning(RL). First, entities and relations are mapped to lowdimensional vector space through TuckER embedding, and the path reasoning process is modeled using RL guided by strategies in the knowledge graph environment. Then, the action pruning mechanism is introduced to reduce the interference of invalid actions for action selection during path walking, and LSTM is used as the memory component to preserve the agent’s historical action trajectory. In this way, the agent can more accurately select valid actions and can complete knowledge reasoning by interaction with the knowledge graph. The experiments on three mainstream large-scale datasets indicate that TuckRL is superior to most of the existing methods, which demonstrates the effectiveness of combining embedding and RL for knowledge reasoning.