Multi-modal knowledge graph inference via media convergence and logic rule

Media convergence works by processing information from different modalities and applying them to different domains.It is difficult for the conventional knowledge graph to utilise multi-media features because the introduction of a large amount of information from other modalities reduces the effectiveness of representation learning and makes knowledge graph inference less effective.To address the issue,an inference method based on Media Convergence and Rule-guided Joint Inference model(MCRJI)has been pro-posed.The authors not only converge multi-media features of entities but also introduce logic rules to improve the accuracy and interpretability of link prediction.First,a multi-headed self-attention approach is used to obtain the attention of different media features of entities during semantic synthesis.Second,logic rules of different lengths are mined from knowledge graph to learn new entity representations.Finally,knowledge graph inference is performed based on representing entities that converge multi-media features.Numerous experimental results show that MCRJI outperforms other advanced baselines in using multi-media features and knowledge graph inference,demonstrating that MCRJI provides an excellent approach for knowledge graph inference with converged multi-media features.

Construction of fault diagnosis system for control rod drive mechanism based on knowledge graph and Bayesian inference

Knowledge graph technology has distinct advantages in terms of fault diagnosis.In this study,the control rod drive mechanism(CRDM)of the liquid fuel thorium molten salt reactor(TMSR-LF1)was taken as the research object,and a fault diagnosis system was proposed based on knowledge graph.The subject–relation–object triples are defined based on CRDM unstructured data,including design specification,operation and maintenance manual,alarm list,and other forms of expert experience.In this study,we constructed a fault event ontology model to label the entity and relationship involved in the corpus of CRDM fault events.A three-layer robustly optimized bidirectional encoder representation from transformers(RBT3)pre-training approach combined with a text convolutional neural network(TextCNN)was introduced to facilitate the application of the constructed CRDM fault diagnosis graph database for fault query.The RBT3-TextCNN model along with the Jieba tool is proposed for extracting entities and recognizing the fault query intent simultaneously.Experiments on the dataset collected from TMSR-LF1 CRDM fault diagnosis unstructured data demonstrate that this model has the potential to improve the effect of intent recognition and entity extraction.Additionally,a fault alarm monitoring module was developed based on WebSocket protocol to deliver detailed information about the appeared fault to the operator automatically.Furthermore,the Bayesian inference method combined with the variable elimination algorithm was proposed to enable the development of a relatively intelligent and reliable fault diagnosis system.Finally,a CRDM fault diagnosis Web interface integrated with graph data visualization was constructed,making the CRDM fault diagnosis process intuitive and effective.

Application of graph neural network and feature information enhancement in relation inference of sparse knowledge graph

At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.

A Survey of Knowledge Graph Construction Using Machine Learning

Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.

A Novel Attack Graph Posterior Inference Model Based on Bayesian Network

Network attack graphs are originally used to evaluate what the worst security state is when a concerned net-work is under attack. Combined with intrusion evidence such like IDS alerts, attack graphs can be further used to perform security state posterior inference (i.e. inference based on observation experience). In this area, Bayesian network is an ideal mathematic tool, however it can not be directly applied for the following three reasons: 1) in a network attack graph, there may exist directed cycles which are never permitted in a Bayesian network, 2) there may exist temporal partial ordering relations among intrusion evidence that can-not be easily modeled in a Bayesian network, and 3) just one Bayesian network cannot be used to infer both the current and the future security state of a network. In this work, we improve an approximate Bayesian posterior inference algorithm–the likelihood-weighting algorithm to resolve the above obstacles. We give out all the pseudocodes of the algorithm and use several examples to demonstrate its benefit. Based on this, we further propose a network security assessment and enhancement method along with a small network scenario to exemplify its usage.

QA-KGNet:一种语言模型驱动的知识图谱问答模型

基于知识图谱的问答系统可以解析用户问题,已成为一种检索知识、自动回答所询问题的有效途径.知识图谱问答系统通常是利用神经程序归纳模型,将自然语言问题转化为逻辑形式,在知识图谱上执行该逻辑形式能够得到答案.然而,使用预训练语言模型和知识图谱的知识问答系统包含两个挑战:(1)给定问答(questionanswering, QA)上下文,需要从大型知识图谱(knowledge graph, KG)中识别相关知识;(2)对QA上下文和KG进行联合推理.基于此,提出一种语言模型驱动的知识图谱问答推理模型QA-KGNet,将QA上下文和KG连接起来形成一个工作图,使用语言模型计算给定QA上下文节点与KG节点的关联度,并使用多头图注意力网络更新节点表示.在Commonsense QA、OpenBookQA和Med QA-USMLE真实数据集上进行实验来评估QA-KGNet的性能,实验结果表明:QA-KGNet优于现有的基准模型,表现出优越的结构化推理能力.

Semantic-aware graph convolution network on multi-hop paths for link prediction

Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack transparency of model prediction principles. In this paper,a new graph convolutional network path semantic-aware graph convolution network(PSGCN) is proposed to achieve modeling the semantic information of multi-hop paths. PSGCN first uses a random walk strategy to obtain all-hop paths in KGs,then captures the semantics of the paths by Word2Sec and long shortterm memory(LSTM) models,and finally converts them into a potential representation for the graph convolution network(GCN) messaging process. PSGCN combines path-based inference methods and graph neural networks to achieve better interpretability and scalability. In addition,to ensure the robustness of the model,the value of the path thresholdKis experimented on the FB15K-237 and WN18RR datasets,and the final results prove the effectiveness of the model.

RotatS:temporal knowledge graph completion based on rotation and scaling in 3D space

As the research of knowledge graph(KG)is deepened and widely used,knowledge graph com-pletion(KGC)has attracted more and more attentions from researchers,especially in scenarios of in-telligent search,social networks and deep question and answer(Q&A).Current research mainly fo-cuses on the completion of static knowledge graphs,and the temporal information in temporal knowl-edge graphs(TKGs)is ignored.However,the temporal information is definitely very helpful for the completion.Note that existing researches on temporal knowledge graph completion are difficult to process temporal information and to integrate entities,relations and time well.In this work,a rotation and scaling(RotatS)model is proposed,which learns rotation and scaling transformations from head entity embedding to tail entity embedding in 3D spaces to capture the information of time and rela-tions in the temporal knowledge graph.The performance of the proposed RotatS model have been evaluated by comparison with several baselines under similar experimental conditions and space com-plexity on four typical knowl good graph completion datasets publicly available online.The study shows that RotatS can achieve good results in terms of prediction accuracy.

基于多邻域感知的石油数据资产图谱实体对齐

实体对齐在自动融合多源异构的石油领域数据资产知识图谱过程中起着至关重要的作用。目前主流基于图神经网络的实体对齐模型多关注实体和图结构的信息,忽略了实体之间的关系、属性与属性值等多邻域的语义信息,在命名规则差异性大、行业特殊、语义实体多的石油领域数据资产知识图谱融合过程中性能一般。提出一种基于图注意力网络改进的多邻域感知网络(MNAN)模型,用于实体对齐。使用基于BERT的多语言预训练模型得到实体及多邻域的初始特征,通过带有Highway Networks的图卷积神经网络聚合邻域实体与图结构特征,利用多邻域感知和实体增强注意力网络聚合实体的多邻域特征,使用最小化基于边际的损失函数训练模型。在石油领域数据资产知识图谱数据集中的2个知识图谱上进行实体对齐实验,实验结果表明,MNAN模型优于所有对比的基于图神经网络实体对齐模型,Hits@1值达86.7%,优于表现最好的对比模型约2.3个百分点。

基于场景理解的施工临边坠落险兆智能识别方法

为更及时、更有效地预防施工临边坠落事故的发生,并弥补现有智能预警方法在场景理解方面的不足,融合深度学习与语义推理,提出1种险兆识别方法。该方法通过neo4j构建险兆知识图谱,将引入轻量级视觉Transformer的YOLOx模型识别工人的险兆行为,设计描述空间关系的IoU计算方法并使用Cypher推理语言进行险兆推理。研究结果表明:施工临边坠落各要素识别的平均精度达91%以上,且IoU计算与险兆推理准确率均为100%,模型识别效果与险兆推理效果较好,该方法总体满足精度和速度的识别要求。研究结果可为实现施工临边坠落险兆行为的精准识别和预警提供参考。

基于多维能力和知识图谱-多层感知机的变压器运行状态画像构建方法

利用大数据和画像技术对电力变压器运行状态进行准确评价有利于保障电力系统的安全稳定运行。针对电力变压器运行状态传统评价方法存在的评价维度过于单一、主观性较强等不足,提出了一种基于多维能力和知识图谱-多层感知机的变压器运行状态画像构建方法。首先,构建了由绝缘水平、负载能力、抗短路能力、能效等级和调压能力五个能力构成的变压器运行状态画像体系;然后,融合知识图谱(knowledge graph,KG)与多层感知机(multilayer perceptron,MLP),建立了一种变压器运行状态画像分析模型;最后,基于某地区1368台110kV变压器的实际运行数据,开展了变压器运行状态画像的实例分析,并与随机森林(random forest,RF)和支持向量机(support vector machine,SVM)方法的画像分析结果进行对比。研究结果表明,所提方法对变压器运行状态画像的准确率达到96.35%,优于RF算法(准确率89%)和SVM算法(准确率77%),为电力变压器的运行状态评价提供了一种新思路。

GNNSched:面向GPU的图神经网络推理任务调度框架

由于频繁的显存访问,图神经网络GNN在GPU上运行时往往资源利用率较低。现有的推理框架由于没有考虑GNN输入的不规则性,直接适用到GNN进行推理任务共置时可能会超出显存容量导致任务失败。对于GNN推理任务,需要根据其输入特点预先分析并发任务的显存占用情况,以确保并发任务在GPU上的成功共置。此外,多租户场景提交的推理任务亟需灵活的调度策略,以满足并发推理任务的服务质量要求。为了解决上述问题,提出了GNNSched,其在GPU上高效管理GNN推理任务的共置运行。具体来说,GNNSched将并发推理任务组织为队列,并在算子粒度上根据成本函数估算每个任务的显存占用情况。GNNSched实现了多种调度策略来生成任务组,这些任务组被迭代地提交到GPU并发执行。实验结果表明,GNNSched能够满足并发GNN推理任务的服务质量并降低推理任务的响应时延。

基于时序图推理的设备剩余使用寿命预测

剩余使用寿命(Remaining useful life,RUL)预测是大型设备故障预测与健康管理(Prognostics and health management,PHM)的重要环节,对于降低设备维修成本和避免灾难性故障具有重要意义.针对RUL预测,首次提出一种基于多变量分析的时序图推理模型(Multivariate similarity temporal knowledge graph,MSTKG),通过捕捉设备各部件的运行状态耦合关系及其变化趋势,挖掘其中蕴含的设备性能退化信息,为寿命预测提供有效依据.首先,设计时序图结构,形式化表达各部件不同工作周期的关联关系.其次,提出联合图卷积神经网络(Convolutional neural network,CNN)和门控循环单元(Gated recurrent unit,GRU)的深度推理网络,建模并学习设备各部件工作状态的时空演化过程,并结合回归分析,得到剩余使用寿命预测结果.最后,与现有预测方法相比,所提方法能够显式建模并利用设备部件耦合关系的变化信息,仿真实验结果验证了该方法的优越性.

基于文本知识增强的问题生成模型

预训练语言模型在大规模训练数据和超大规模算力的基础上,能够从非结构化的文本数据中学到大量的知识。针对三元组包含信息有限的问题,提出利用预训练语言模型丰富知识的问题生成方法。首先,利用预训练语言模型中丰富的知识增强三元组信息,设计文本知识生成器,将三元组中的信息转化为子图描述,丰富三元组的语义;然后,使用问题类型预测器预测疑问词,准确定位答案所在的领域,从而生成语义正确的问题,更好地控制问题生成的效果;最后,设计一种受控生成框架对关键实体和疑问词进行约束,保证关键实体和疑问词同时出现在问题中,使生成的问题更加准确。在公开数据集WebQuestion和PathQuestion中验证所提模型的性能。实验结果表明,与现有模型LFKQG相比,所提模型的BLUE-4、METEOR、ROUGE-L指标在WebQuestion数据集上分别提升0.28、0.16、0.22个百分点,在PathQuestion数据集上分别提升0.8、0.39、0.46个百分点。

基于知识感知和跨层次对比学习的推荐方法

知识图谱(KG)作为一种辅助信息能够有效提高推荐模型的推荐质量,但现有的基于图神经网络(GNN)的知识感知推荐模型存在节点信息利用不均衡问题。为此,提出一种基于知识感知和跨层次对比学习的推荐方法(KCCL)。所提方法在GNN的知识感知推荐模型基础上引入对比学习范式,以缓解稀疏的交互数据和嘈杂的KG在信息聚合时节点间依赖的关系偏离真实表示导致节点信息利用不均衡的问题。首先,将用户–物品交互图和物品知识图整合为一个异质图,并通过基于图注意力机制的GNN实现用户和物品的节点表示;其次,在信息传播聚合层中加入一致的噪声进行数据增强,得到不同阶层的节点表示,并将获得的最外层节点表示与最内层节点表示进行跨层次对比学习;最后,联合优化推荐监督任务和对比学习辅助任务,得到最终各节点表示。在DBbook2014和MovieLens-1m数据集上的实验结果显示,相较于次优对比方法,KCCL的Recall@10分别提升了3.66%和0.66%,NDCG@10分别提升了3.57%和3.29%,验证了KCCL的有效性。

面向知识图谱链接预测任务的解释子图生成模型

近年来图神经网络(GNN)发展迅速,相关模型在知识图谱链接预测任务上的性能显著提升。为解释性能提升的原因,研究人员需要提取GNN学习到的子图模式。然而现有GNN解释器在知识图谱这类典型多关系(multi-relation)图数据场景下的解释准确性尚未被验证,且相关工具尚未实现,导致解释子图提取困难。针对该问题,提出一种将多关系的知识图谱转换为单关系(uni-relational)图的知识图谱链接预测模型,该模型通过将知识图谱中的实体组合为新的节点,并将关系作为新节点的特征,生成只有单一关系的新图,并在新图上训练去噪自编码器使其获得链接预测能力,最后使用GNN解释器生成子图解释。在三个基准数据集上的实验表明,与不进行转换的GraIL相比,基于单关系转换的链接预测模型的相对AUC指标提升显著。最后,该模型选取FB15K-237数据集进行解释子图提取实验,验证了模型在直接提取链接预测解释方面的有效性。

一种基于知识图谱的用户多偏好推荐系统

提出了一种基于知识图谱(KG)的用户多偏好(MPKG)推荐系统,从用户关系级、实体级和细粒度高阶用户三种不同的视角建模用户的偏好.首先,将KG中关系向量组合,构建关系级意图,并通过独立性将不同意图之间的差异最大化,由关系级意图来指导学习关系级偏好;然后,根据用户交互实体的频率构建实体偏好图(EPG),并学习用户的实体级偏好;接着,分别使用关系级意图和实体级偏好来指导模型学习用户的表示;此外,还直接从KG中构建关系实体信息流,用于用户的表示,挖掘用户的高阶细粒度偏好.在两个基准数据集上进行实验,实验结果验证了该方法的有效性和可行性.

Knowledge enhanced graph inference network based entity-relation extraction and knowledge graph construction for industrial domain

With the escalating complexity in production scenarios, vast amounts of production information are retained within enterprises in the industrial domain. Probing questions of how to meticulously excavate value from complex document information and establish coherent information links arise. In this work, we present a framework for knowledge graph construction in the industrial domain, predicated on knowledge-enhanced document-level entity and relation extraction. This approach alleviates the shortage of annotated data in the industrial domain and models the interplay of industrial documents. To augment the accuracy of named entity recognition, domain-specific knowledge is incorporated into the initialization of the word embedding matrix within the bidirectional long short-term memory conditional random field (BiLSTM-CRF) framework. For relation extraction, this paper introduces the knowledge-enhanced graph inference (KEGI) network, a pioneering method designed for long paragraphs in the industrial domain. This method discerns intricate interactions among entities by constructing a document graph and innovatively integrates knowledge representation into both node construction and path inference through TransR. On the application stratum, BiLSTM-CRF and KEGI are utilized to craft a knowledge graph from a knowledge representation model and Chinese fault reports for a steel production line, specifically SPOnto and SPFRDoc. The F1 value for entity and relation extraction has been enhanced by 2% to 6%. The quality of the extracted knowledge graph complies with the requirements of real-world production environment applications. The results demonstrate that KEGI can profoundly delve into production reports, extracting a wealth of knowledge and patterns, thereby providing a comprehensive solution for production management.

基于双层图注意力网络的邻域信息聚合实体对齐方法

针对知识图谱中存在部分属性信息对实体对齐任务影响程度不一致以及实体的邻域信息重要程度不一致的问题,提出了一种结合双层图注意力网络的邻域信息聚合实体对齐(two-layer graph attention network entity alignment,TGAEA)方法。该方法采用双层图神经网络,首先利用第一层网络对实体属性进行注意力系数计算,降低无用属性对实体对齐的影响;随后,结合第二层网络对实体名称、关系和结构等信息进行特征加权,以区分实体邻域信息的重要性;最后,借助自举方法扩充种子实体对,并结合邻域信息相似度矩阵进行实体距离度量。实验表明,在DWY100K数据集上,TGAEA模型相较于当前基线模型,hit@1、hit@10和MRR指标分别提升了4.18%、4.81%和5%,证明了双层图注意力网络在邻域信息聚合实体对齐方面的显著效果。

基于不确定知识图谱嵌入的多关系近似推理模型

针对大规模知识图谱(KG)的不确定性嵌入模型中无法对多种逻辑关系进行近似推理的问题,提出一种基于不确定KG嵌入(UKGE)的多关系近似推理模型UDConEx(Uncertainty DistMult(Distance Multiplicative) and complex Convolution Embedding)。首先,UDConEx结合DistMult和ComplEx(Complex Embedding)模型的特点,使得UDConEx具有推理对称与非对称关系的能力;其次,UDConEx采用卷积神经网络(CNN)捕获不确定性KG中的交互信息,使它具有推理逆关系和传递关系的能力;最后,UDConEx利用神经网络对KG的不确定信息进行置信度学习,在UKGE空间中可以进行近似推理。在CN15k、NL27k和PPI5k这3个公开数据集上的实验结果表明,相较于MUKGE(Multiplex UKGE)模型,UDConEx在CN15k、NL27k和PPI5k的置信度预测任务中平均绝对误差(MAE)分别降低了6.3%,30.1%和44.9%;在关系事实排名任务中,基于线性的归一化折损累计增益(NDCG)在CN15k和NL27k数据集中分别提升了5.8%和2.6%;在多关系近似推理任务中验证了UDConEx具有多种逻辑关系的近似推理能力。UDConEx弥补了传统嵌入模型无法进行置信度预测的不足,实现了对多种逻辑关系的近似推理,具有更精确、具有可解释性的不确定性知识图谱推理能力。