Hyperbolic hierarchical graph attention network for knowledge graph completion

Utilizing graph neural networks for knowledge embedding to accomplish the task of knowledge graph completion(KGC)has become an important research area in knowledge graph completion.However,the number of nodes in the knowledge graph increases exponentially with the depth of the tree,whereas the distances of nodes in Euclidean space are second-order polynomial distances,whereby knowledge embedding using graph neural networks in Euclidean space will not represent the distances between nodes well.This paper introduces a novel approach called hyperbolic hierarchical graph attention network(H2GAT)to rectify this limitation.Firstly,the paper conducts knowledge representation in the hyperbolic space,effectively mitigating the issue of exponential growth of nodes with tree depth and consequent information loss.Secondly,it introduces a hierarchical graph atten-tion mechanism specifically designed for the hyperbolic space,allowing for enhanced capture of the network structure inherent in the knowledge graph.Finally,the efficacy of the proposed H2GAT model is evaluated on benchmark datasets,namely WN18RR and FB15K-237,thereby validating its effectiveness.The H2GAT model achieved 0.445,0.515,and 0.586 in the Hits@1,Hits@3 and Hits@10 metrics respectively on the WN18RR dataset and 0.243,0.367 and 0.518 on the FB15K-237 dataset.By incorporating hyperbolic space embedding and hierarchical graph attention,the H2GAT model successfully addresses the limitations of existing hyperbolic knowledge embedding models,exhibiting its competence in knowledge graph completion tasks.

Future Event Prediction Based on Temporal Knowledge Graph Embedding

Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling complex systems.Different types of events continually occur,which are often related to historical and concurrent events.In this paper,we formalize the future event prediction as a temporal knowledge graph reasoning problem.Most existing studies either conduct reasoning on static knowledge graphs or assume knowledges graphs of all timestamps are available during the training process.As a result,they cannot effectively reason over temporal knowledge graphs and predict events happening in the future.To address this problem,some recent works learn to infer future events based on historical eventbased temporal knowledge graphs.However,these methods do not comprehensively consider the latent patterns and influences behind historical events and concurrent events simultaneously.This paper proposes a new graph representation learning model,namely Recurrent Event Graph ATtention Network(RE-GAT),based on a novel historical and concurrent events attention-aware mechanism by modeling the event knowledge graph sequence recurrently.More specifically,our RE-GAT uses an attention-based historical events embedding module to encode past events,and employs an attention-based concurrent events embedding module to model the associations of events at the same timestamp.A translation-based decoder module and a learning objective are developed to optimize the embeddings of entities and relations.We evaluate our proposed method on four benchmark datasets.Extensive experimental results demonstrate the superiority of our RE-GAT model comparing to various base-lines,which proves that our method can more accurately predict what events are going to happen.

Extrapolation over temporal knowledge graph via hyperbolic embedding

Predicting potential facts in the future,Temporal Knowledge Graph(TKG)extrapolation remains challenging because of the deep dependence between the temporal association and semantic patterns of facts.Intuitively,facts(events)that happened at different timestamps have different influences on future events,which can be attributed to a hierarchy among not only facts but also relevant entities.Therefore,it is crucial to pay more attention to important entities and events when forecasting the future.However,most existing methods focus on reasoning over temporally evolving facts or mining evolutional patterns from known facts,which may be affected by the diversity and variability of the evolution,and they might fail to attach importance to facts that matter.Hyperbolic geometry was proved to be effective in capturing hierarchical patterns among data,which is considered to be a solution for modelling hierarchical relations among facts.To this end,we propose ReTIN,a novel model integrating real-time influence of historical facts for TKG reasoning based on hyperbolic geometry,which provides low-dimensional embeddings to capture latent hierarchical structures and other rich semantic patterns of the existing TKG.Considering both real-time and global features of TKG boosts the adaptation of ReTIN to the ever-changing dynamics and inherent constraints.Extensive experiments on benchmarks demonstrate the superiority of ReTIN over various baselines.The ablation study further supports the value of exploiting temporal information.

Learning Context-based Embeddings for Knowledge Graph Completion

Purpose:Due to the incompleteness nature of knowledge graphs(KGs),the task of predicting missing links between entities becomes important.Many previous approaches are static,this posed a notable problem that all meanings of a polysemous entity share one embedding vector.This study aims to propose a polysemous embedding approach,named KG embedding under relational contexts(ContE for short),for missing link prediction.Design/methodology/approach:ContE models and infers different relationship patterns by considering the context of the relationship,which is implicit in the local neighborhood of the relationship.The forward and backward impacts of the relationship in ContE are mapped to two different embedding vectors,which represent the contextual information of the relationship.Then,according to the position of the entity,the entity’s polysemous representation is obtained by adding its static embedding vector to the corresponding context vector of the relationship.Findings:ContE is a fully expressive,that is,given any ground truth over the triples,there are embedding assignments to entities and relations that can precisely separate the true triples from false ones.ContE is capable of modeling four connectivity patterns such as symmetry,antisymmetry,inversion and composition.Research limitations:ContE needs to do a grid search to find best parameters to get best performance in practice,which is a time-consuming task.Sometimes,it requires longer entity vectors to get better performance than some other models.Practical implications:ContE is a bilinear model,which is a quite simple model that could be applied to large-scale KGs.By considering contexts of relations,ContE can distinguish the exact meaning of an entity in different triples so that when performing compositional reasoning,it is capable to infer the connectivity patterns of relations and achieves good performance on link prediction tasks.Originality/value:ContE considers the contexts of entities in terms of their positions in triples and the relationships they link to.It decomposes a relation vector into two vectors,namely,forward impact vector and backward impact vector in order to capture the relational contexts.ContE has the same low computational complexity as TransE.Therefore,it provides a new approach for contextualized knowledge graph embedding.

Knowledge Graph Representation Learning Based on Automatic Network Search for Link Prediction

Link prediction,also known as Knowledge Graph Completion(KGC),is the common task in Knowledge Graphs(KGs)to predict missing connections between entities.Most existing methods focus on designing shallow,scalable models,which have less expressive than deep,multi-layer models.Furthermore,most operations like addition,matrix multiplications or factorization are handcrafted based on a few known relation patterns in several wellknown datasets,such as FB15k,WN18,etc.However,due to the diversity and complex nature of real-world data distribution,it is inherently difficult to preset all latent patterns.To address this issue,we proposeKGE-ANS,a novel knowledge graph embedding framework for general link prediction tasks using automatic network search.KGEANS can learn a deep,multi-layer effective architecture to adapt to different datasets through neural architecture search.In addition,the general search spacewe designed is tailored forKGtasks.We performextensive experiments on benchmark datasets and the dataset constructed in this paper.The results show that our KGE-ANS outperforms several state-of-the-art methods,especially on these datasets with complex relation patterns.

Knowledge Graph Embedding Based on Adaptive Negative Sampling

Knowledge graph embedding aims at embedding entities and relations in a knowledge graph into a continuous, dense, low-dimensional and realvalued vector space. Among various embedding models appeared in recent years, translation-based models such as TransE, TransH and TransR achieve state-of-the-art performance. However, in these models, negative triples used for training phase are generated by replacing each positive entity in positive triples with negative entities from the entity set with the same probability;as a result, a large number of invalid negative triples will be generated and used in the training process. In this paper, a method named adaptive negative sampling (ANS) is proposed to generate valid negative triples. In this method, it first divided all the entities into a number of groups which consist of similar entities by some clustering algorithms such as K-Means. Then, corresponding to each positive triple, the head entity was replaced by a negative entity from the cluster in which the head entity was located and the tail entity was replaced in a similar approach. As a result, it generated a set of high-quality negative triples which benefit for improving the effectiveness of embedding models. The ANS method was combined with the TransE model and the resulted model was named as TransE-ANS. Experimental results show that TransE-ANS achieves significant improvement in the link prediction task.

Uncertain knowledge graph embedding:an effective method combining multi-relation and multi-path

Uncertain Knowledge Graphs(UKGs)are used to characterize the inherent uncertainty of knowledge and have a richer semantic structure than deterministic knowledge graphs.The research on the embedding of UKG has only recently begun,Uncertain Knowledge Graph Embedding(UKGE)model has a certain effect on solving this problem.However,there are still unresolved issues.On the one hand,when reasoning the confidence of unseen relation facts,the introduced probabilistic soft logic cannot be used to combine multi-path and multi-step global information,leading to information loss.On the other hand,the existing UKG embedding model can only model symmetric relation facts,but the embedding problem of asymmetric relation facts has not be addressed.To address the above issues,a Multiplex Uncertain Knowledge Graph Embedding(MUKGE)model is proposed in this paper.First,to combine multiple information and achieve more accurate results in confidence reasoning,the Uncertain ResourceRank(URR)reasoning algorithm is introduced.Second,the asymmetry in the UKG is defined.To embed asymmetric relation facts of UKG,a multi-relation embedding model is proposed.Finally,experiments are carried out on different datasets via 4 tasks to verify the effectiveness of MUKGE.The results of experiments demonstrate that MUKGE can obtain better overall performance than the baselines,and it helps advance the research on UKG embedding.

Collective Entity Alignment for Knowledge Fusion of Power Grid Dispatching Knowledge Graphs

Knowledge graphs(KGs)have been widely accepted as powerful tools for modeling the complex relationships between concepts and developing knowledge-based services.In recent years,researchers in the field of power systems have explored KGs to develop intelligent dispatching systems for increasingly large power grids.With multiple power grid dispatching knowledge graphs(PDKGs)constructed by different agencies,the knowledge fusion of different PDKGs is useful for providing more accurate decision supports.To achieve this,entity alignment that aims at connecting different KGs by identifying equivalent entities is a critical step.Existing entity alignment methods cannot integrate useful structural,attribute,and relational information while calculating entities’similarities and are prone to making many-to-one alignments,thus can hardly achieve the best performance.To address these issues,this paper proposes a collective entity alignment model that integrates three kinds of available information and makes collective counterpart assignments.This model proposes a novel knowledge graph attention network(KGAT)to learn the embeddings of entities and relations explicitly and calculates entities’similarities by adaptively incorporating the structural,attribute,and relational similarities.Then,we formulate the counterpart assignment task as an integer programming(IP)problem to obtain one-to-one alignments.We not only conduct experiments on a pair of PDKGs but also evaluate o ur model on three commonly used cross-lingual KGs.Experimental comparisons indicate that our model outperforms other methods and provides an effective tool for the knowledge fusion of PDKGs.

Joint learning based on multi-shaped filters for knowledge graph completion

To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge graph completion(KGC).Related research work has shown the superiority of convolutional neural networks(CNNs)in extracting semantic features of triple embeddings.However,these researches use only one single-shaped filter and fail to extract semantic features of different granularity.To solve this problem,ConvKG exploits multi-shaped filters to co-convolute on the triple embeddings,joint learning semantic features of different granularity.Different shaped filters cover different sizes on the triple embeddings and capture pairwise interactions of different granularity among triple elements.Experimental results confirm the strength of joint learning,and compared with state-of-the-art CNN-based KGC models,ConvKG achieves the better mean rank(MR)and Hits@10 metrics on dataset WN18 RR,and the better MR on dataset FB15k-237.

Deep Knowledge Tracing Embedding Neural Network for Individualized Learning

Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Available knowledge tracing models have the problem that the assessments are not directly used in the predictions.To make full use of the assessments during predictions,a novel model,named deep knowledge tracing embedding neural network(DKTENN),is proposed in this work.DKTENN is a synthesis of deep knowledge tracing(DKT)and knowledge graph embedding(KGE).DKT utilizes sophisticated long short-term memory(LSTM)to assess the users and track the mastery of skills according to the users'interaction sequences with skill-level tags,and KGE is applied to predict the probability on the basis of both the embedded problems and DKT's assessments.DKTENN outperforms performance factors analysis and the other knowledge tracing models based on deep learning in the experiments.

Fusion Recommendation System Based on Collaborative Filtering and Knowledge Graph

The recommendation algorithm based on collaborative filtering is currently the most successful recommendation method. It recommends items to theuser based on the known historical interaction data of the target user. Furthermore,the combination of the recommended algorithm based on collaborative filtrationand other auxiliary knowledge base is an effective way to improve the performance of the recommended system, of which the Co-Factorization Model(CoFM) is one representative research. CoFM, a fusion recommendation modelcombining the collaborative filtering model FM and the graph embeddingmodel TransE, introduces the information of many entities and their relationsin the knowledge graph into the recommendation system as effective auxiliaryinformation. It can effectively improve the accuracy of recommendations andalleviate the problem of sparse user historical interaction data. Unfortunately,the graph-embedded model TransE used in the CoFM model cannot solve the1-N, N-1, and N-N problems well. To tackle this problem, a novel fusion recommendation model Joint Factorization Machines and TransH Model (JFMH) isproposed, which improves CoFM by replacing the TransE model with TransHmodel. A large number of experiments on two widely used benchmark data setsshow that compared with CoFM, JFMH has improved performance in terms ofitem recommendation and knowledge graph completion, and is more competitivethan multiple baseline methods.

Construction of well logging knowledge graph and intelligent identification method of hydrocarbon-bearing formation

Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting characteristic parameters describing HBF in multiple dimensions and multiple scales;(2)showing the characteristic parameter-related entities,relationships,and attributes as vectors via graph embedding technique;(3)intelligently identifying HBF;(4)seamlessly integrating expertise into the intelligent computing to establish the assessment system and ranking algorithm for potential pay recommendation.Taking 547 wells encountered the low porosity and low permeability Chang 6 Member of Triassic in the Jiyuan Block of Ordos Basin,NW China as objects,80%of the wells were randomly selected as the training dataset and the remainder as the validation dataset.The KPNFE prediction results on the validation dataset had a coincidence rate of 94.43%with the expert interpretation results and a coincidence rate of 84.38%for all the oil testing layers,which is 13 percentage points higher in accuracy and over 100 times faster than the primary conventional interpretation.In addition,a number of potential pays likely to produce industrial oil were recommended.The KPNFE model effectively inherits,carries forward and improves the expert knowledge,nicely solving the robustness problem in HBF identification.The KPNFE,with good interpretability and high accuracy of computation results,is a powerful technical means for efficient and high-quality well logging re-evaluation of old wells in mature oilfields.

Research on knowledge reasoning of TCM based on knowledge graphs

With the widespread use of Internet,the amount of data in the field of traditional Chinese medicine(TCM)is growing exponentially.Consequently,there is much attention on the collection of useful knowledge as well as its effective organization and expression.Knowledge graphs have thus emerged,and knowledge reasoning based on this tool has become one of the hot spots of research.This paper first presents a brief introduction to the development of knowledge graphs and knowledge reasoning,and explores the significance of knowledge reasoning.Secondly,the mainstream knowledge reasoning methods,including knowledge reasoning based on traditional rules,knowledge reasoning based on distributed feature representation,and knowledge reasoning based on neural networks are introduced.Then,using stroke as an example,the knowledge reasoning methods are expounded,the principles and characteristics of commonly used knowledge reasoning methods are summarized,and the research and applications of knowledge reasoning techniques in TCM in recent years are sorted out.Finally,we summarize the problems faced in the development of knowledge reasoning in TCM,and put forward the importance of constructing a knowledge reasoning model suitable for the field of TCM.

基于知识图谱嵌入与深度学习的药物不良反应预测

识别药物潜在的不良反应,有助于辅助医生进行临床用药决策。针对以往研究的特征高维稀疏、需要为每种不良反应构建独立预测模型且预测精度较低的问题,本文开发一种基于知识图谱嵌入和深度学习的药物不良反应预测模型,能够对实验所覆盖的不良反应进行统一预测。一方面,知识图谱及其嵌入技术能够融合药物之间的关联信息,缓解特征矩阵高维稀疏的不足;另一方面,深度学习的高效训练能力能够提升模型的预测精度。本文使用药物特征数据构建药物不良反应知识图谱;通过分析不同嵌入策略下知识图谱的嵌入效果,选择最佳嵌入策略以获得样本向量;然后构建卷积神经网络模型对不良反应进行预测。结果表明,在DistMult嵌入模型和400维嵌入策略下,卷积神经网络模型预测效果最佳;重复实验的准确率、F_1分数、召回率和曲线下面积的平均值分别为0.887、0.890、0.913和0.957,优于文献报道中的方法。所得预测模型具有较好的预测精度和稳定性,可以为安全用药提供有效参考。

基于问题与关系嵌入空间对齐的知识图谱问答

为解决基于嵌入的知识图谱问答中,因采用不同的训练模型导致问题嵌入与知识图谱嵌入处于不同语义空间的问题,提出一种知识图谱问答模型。将关系转换为手工构造的自然语言问题,通过神经网络训练问题的嵌入表示与知识图谱的关系嵌入表示尽可能靠近;通过训练集中的问题对神经网络参数进行微调,使答案获得最高评分。在WebquestionSP数据集上的实验结果表明,相较于EmbedKGQA,所提方法的hits@1指标提高了10.5个百分点;在缺失50%三元组的情况下hits@1指标提高了9.9个百分点。

k阶采样和图注意力网络的知识图谱表示模型

知识图谱表示(KGE)旨在将知识图谱中的实体和关系映射到低维度向量空间而获得其向量表示。现有的KGE模型只考虑一阶近邻,这影响了知识图谱中推理和预测任务的准确性。为了解决这一问题,提出了一种基于k阶采样算法和图注意力网络的KGE模型。k阶采样算法通过聚集剪枝子图中的k阶邻域来获取中心实体的邻居特征。引入图注意力网络来学习中心实体邻居的注意力值,通过邻居特征加权和得到新的实体向量表示。利用ConvKB作为解码器来分析三元组的全局表示特征。在WN18RR、FB15k-237、NELL-995、Kinship数据集上的评价实验表明,该模型在链接预测任务上的性能明显优于最新的模型。此外,还讨论了阶数k和采样系数b的改变对模型命中率的影响。

面向方面情感分析的多通道增强图卷积网络

传统的基于单通道的特征提取方式,仅使用单一的依赖关系捕获特征,忽略单词间的语义相似性与依赖关系类型信息。尽管基于图卷积网络进行方面情感分析的方法已经取得一定成效,但始终难以同时聚合节点的语义信息和句法结构特征,在整个迭代训练过程中最初的语义特征会逐渐遗失,影响句子最终的情感分类效果。由于缺乏先验知识会导致模型对相关情感词的误解,因此需要引入外部知识来丰富文本信息。目前,如何利用图神经网络(GNN)融合句法和语义特征的方式仍值得深入研究。针对上述问题,该文提出一种多通道增强图卷积网络模型。首先,通过对情感知识和依赖类型增强的句法图进行图卷积操作,得到基于语法的两种表示,与经过多头注意力和图卷积学习到的语义表示进行融合,使多通道的特征能够互补学习。实验结果表明,在5个公开数据集上,准确率和宏F1值优于基准模型。由此可见,依赖类型和情感知识均对增强句法图有重要影响,表明融合语义信息与句法结构的有效性。

多源知识图谱事件知识融合方法研究

以事件为中心的动态知识对事件预测等应用至关重要,但现有知识图谱主要关注以实体为中心的静态知识,难以满足需求。本文提出一种融合多源知识的高质量事件知识图谱构造方法,首先定义全局事件模式,利用标签类别从源知识图谱中提取事件知识并构造临时事件知识图,提出关系扩充规则对临时事件知识图进行扩充,改进实体对齐Attce模型,基于TransD模型对多个临时事件知识图进行联合嵌入学习,以提高实体对齐和冲突发现的效率;利用事件描述完整度计算源知识图谱可信度,发生冲突时作为判别标准进行处理。经过在真实数据集上的实验,验证了该方法的准确性和有效性。

基于双向注意力的图神经推荐算法研究

目的推荐系统主要目标是分析用户的历史行为,以预测用户最感兴趣的项目,随着电子商务和在线服务的日益发展,个性化推荐已成为当今信息传播的基石。基于知识图谱的神经网络推荐通过构建知识图作为输入,可以很好地将节点信息和拓扑结构相结合进行预测,在推荐准确性方面已经证明了良好的结果。然而,现有方法较少考虑图结构中存在的对称关系和信息聚合时梯度消失问题。方法本文提出一种基于知识图谱和神经网络结合的双向注意力机制推荐算法(BGANR),首先将图神经网络与对称注意力机制相结合;然后在不增加额外数据集中维度的情况下,通过双向对称嵌入翻译模型获取用户-项目间的高阶关系,实现对知识图谱中用户-项目信息进行特征的嵌入表示,使注意力机制在决策权重时考虑的关系更全面;其次,基于图神经网络在对节点和邻居信息训练过程中,采用多通道激活函数针对不同的高阶关系进行修正,从而增加反馈的信息量,避免训练过程中的过拟合问题。结果仿真实验结果表明,在Last-FM数据中Recall、NDCG指标与经典模型最好的结果相比,分别提高了2.56%、1.96%。结论BGANR不仅能够实现双向探索高阶连通性,而且在捕捉有效的协同信号的同时能够实现高效信息传递。