The growing prevalence of knowledge reasoning using knowledge graphs(KGs)has substantially improved the accuracy and efficiency of intelligent medical diagnosis.However,current models primarily integrate electronic me...The growing prevalence of knowledge reasoning using knowledge graphs(KGs)has substantially improved the accuracy and efficiency of intelligent medical diagnosis.However,current models primarily integrate electronic medical records(EMRs)and KGs into the knowledge reasoning process,ignoring the differing significance of various types of knowledge in EMRs and the diverse data types present in the text.To better integrate EMR text information,we propose a novel intelligent diagnostic model named the Graph ATtention network incorporating Text representation in knowledge reasoning(GATiT),which comprises text representation,subgraph construction,knowledge reasoning,and diagnostic classification.In the text representation process,GATiT uses a pre-trained model to obtain text representations of the EMRs and additionally enhances embeddings by including chief complaint information and numerical information in the input.In the subgraph construction process,GATiT constructs text subgraphs and disease subgraphs from the KG,utilizing EMR text and the disease to be diagnosed.To differentiate the varying importance of nodes within the subgraphs features such as node categories,relevance scores,and other relevant factors are introduced into the text subgraph.Themessage-passing strategy and attention weight calculation of the graph attention network are adjusted to learn these features in the knowledge reasoning process.Finally,in the diagnostic classification process,the interactive attention-based fusion method integrates the results of knowledge reasoning with text representations to produce the final diagnosis results.Experimental results on multi-label and single-label EMR datasets demonstrate the model’s superiority over several state-of-theart methods.展开更多
Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information ...Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.展开更多
Link prediction,also known as Knowledge Graph Completion(KGC),is the common task in Knowledge Graphs(KGs)to predict missing connections between entities.Most existing methods focus on designing shallow,scalable models...Link prediction,also known as Knowledge Graph Completion(KGC),is the common task in Knowledge Graphs(KGs)to predict missing connections between entities.Most existing methods focus on designing shallow,scalable models,which have less expressive than deep,multi-layer models.Furthermore,most operations like addition,matrix multiplications or factorization are handcrafted based on a few known relation patterns in several wellknown datasets,such as FB15k,WN18,etc.However,due to the diversity and complex nature of real-world data distribution,it is inherently difficult to preset all latent patterns.To address this issue,we proposeKGE-ANS,a novel knowledge graph embedding framework for general link prediction tasks using automatic network search.KGEANS can learn a deep,multi-layer effective architecture to adapt to different datasets through neural architecture search.In addition,the general search spacewe designed is tailored forKGtasks.We performextensive experiments on benchmark datasets and the dataset constructed in this paper.The results show that our KGE-ANS outperforms several state-of-the-art methods,especially on these datasets with complex relation patterns.展开更多
At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production ...At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.展开更多
With the popularity of online learning in educational settings, knowledge tracing(KT) plays an increasingly significant role. The task of KT is to help students learn more effectively by predicting their next mastery ...With the popularity of online learning in educational settings, knowledge tracing(KT) plays an increasingly significant role. The task of KT is to help students learn more effectively by predicting their next mastery of knowledge based on their historical exercise sequences. Nowadays, many related works have emerged in this field, such as Bayesian knowledge tracing and deep knowledge tracing methods. Despite the progress that has been made in KT, existing techniques still have the following limitations: 1) Previous studies address KT by only exploring the observational sparsity data distribution, and the counterfactual data distribution has been largely ignored. 2) Current works designed for KT only consider either the entity relationships between questions and concepts, or the relations between two concepts, and none of them investigates the relations among students, questions, and concepts, simultaneously, leading to inaccurate student modeling. To address the above limitations,we propose a graph counterfactual augmentation method for knowledge tracing. Concretely, to consider the multiple relationships among different entities, we first uniform students, questions, and concepts in graphs, and then leverage a heterogeneous graph convolutional network to conduct representation learning.To model the counterfactual world, we conduct counterfactual transformations on students’ learning graphs by changing the corresponding treatments and then exploit the counterfactual outcomes in a contrastive learning framework. We conduct extensive experiments on three real-world datasets, and the experimental results demonstrate the superiority of our proposed Graph CA method compared with several state-of-the-art baselines.展开更多
A knowledge graph consists of a set of interconnected typed entities and their attributes,which shows a better performance to organize,manage and understand knowledge.However,because knowledge graphs contain a lot of ...A knowledge graph consists of a set of interconnected typed entities and their attributes,which shows a better performance to organize,manage and understand knowledge.However,because knowledge graphs contain a lot of knowledge triples,it is difficult to directly display to researchers.Semantic Link Network is an attempt,and it can deal with the construction,representation and reasoning of semantics naturally.Based on the Semantic Link Network,this paper explores the representation and construction of knowledge graph,and develops an academic knowledge graph prototype system to realize the representation,construction and visualization of knowledge graph.展开更多
Objective To study the research status,research hotspots and development trends in the field of real-world data(RWD)through social network analysis and knowledge graph analysis.Methods RWD of the past 10 years were re...Objective To study the research status,research hotspots and development trends in the field of real-world data(RWD)through social network analysis and knowledge graph analysis.Methods RWD of the past 10 years were retrieved,and literature metrological analysis was made by using UCINET and CiteSpace from CNKI.Results and Conclusion The frequency and centrality of related keywords such as real-world study,hospital information system(HIS),drug combination,data mining and TCM are high.The clusters labeled as clinical medication and RWD contain more keywords.In recent 4 years,there are more articles involving the keywords of data specification,data authenticity,data security and information security.Among them,compound Kushen injection,HIS database and RWD are the top three keywords.It is a long-term research hotspot for Chinese and western medicine to use HIS to study clinical medication,clinical characteristics,diseases and injections.Besides,the research of RWD database has changed from construction to standardized collection and governance,which can make RWD effective.Data authenticity,data security and information security will become the new hotspots in the research of RWD.展开更多
Along with the development of information technologies such as mobile Internet,information acquisition technology,cloud computing and big data technology,the traditional knowledge engineering and knowledge-based softw...Along with the development of information technologies such as mobile Internet,information acquisition technology,cloud computing and big data technology,the traditional knowledge engineering and knowledge-based software engineering have undergone fundamental changes where the network plays an increasingly important role.Within this context,it is required to develop new methodologies as well as technical tools for network-based knowledge representation,knowledge services and knowledge engineering.Obviously,the term“network”has different meanings in different scenarios.Meanwhile,some breakthroughs in several bottleneck problems of complex networks promote the developments of the new methodologies and technical tools for network-based knowledge representation,knowledge services and knowledge engineering.This paper first reviews some recent advances on complex networks,and then,in conjunction with knowledge graph,proposes a framework of networked knowledge which models knowledge and its relationships with the perspective of complex networks.For the unique advantages of deep learning in acquiring and processing knowledge,this paper reviews its development and emphasizes the role that it played in the development of knowledge engineering.Finally,some challenges and further trends are discussed.展开更多
Due to the many types of distributed denial-of-service attacks(DDoS)attacks and the large amount of data generated,it becomes a chal-lenge to manage and apply the malicious behavior knowledge generated by DDoS attacks...Due to the many types of distributed denial-of-service attacks(DDoS)attacks and the large amount of data generated,it becomes a chal-lenge to manage and apply the malicious behavior knowledge generated by DDoS attacks.We propose a malicious behavior knowledge base framework for DDoS attacks,which completes the construction and application of a multi-domain malicious behavior knowledge base.First,we collected mali-cious behavior traffic generated by five mainstream DDoS attacks.At the same time,we completed the knowledge collection mechanism through data pre-processing and dataset design.Then,we designed a malicious behavior category graph and malicious behavior structure graph for the characteristic information and spatial structure of DDoS attacks and completed the knowl-edge learning mechanism using a graph neural network model.To protect the data privacy of multiple multi-domain malicious behavior knowledge bases,we implement the knowledge-sharing mechanism based on federated learning.Finally,we store the constructed knowledge graphs,graph neural network model,and Federated model into the malicious behavior knowledge base to complete the knowledge management mechanism.The experimental results show that our proposed system architecture can effectively construct and apply the malicious behavior knowledge base,and the detection capability of multiple DDoS attacks occurring in the network reaches above 0.95,while there exists a certain anti-interference capability for data poisoning cases.展开更多
Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack...Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack transparency of model prediction principles. In this paper,a new graph convolutional network path semantic-aware graph convolution network(PSGCN) is proposed to achieve modeling the semantic information of multi-hop paths. PSGCN first uses a random walk strategy to obtain all-hop paths in KGs,then captures the semantics of the paths by Word2Sec and long shortterm memory(LSTM) models,and finally converts them into a potential representation for the graph convolution network(GCN) messaging process. PSGCN combines path-based inference methods and graph neural networks to achieve better interpretability and scalability. In addition,to ensure the robustness of the model,the value of the path thresholdKis experimented on the FB15K-237 and WN18RR datasets,and the final results prove the effectiveness of the model.展开更多
To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge gra...To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge graph completion(KGC).Related research work has shown the superiority of convolutional neural networks(CNNs)in extracting semantic features of triple embeddings.However,these researches use only one single-shaped filter and fail to extract semantic features of different granularity.To solve this problem,ConvKG exploits multi-shaped filters to co-convolute on the triple embeddings,joint learning semantic features of different granularity.Different shaped filters cover different sizes on the triple embeddings and capture pairwise interactions of different granularity among triple elements.Experimental results confirm the strength of joint learning,and compared with state-of-the-art CNN-based KGC models,ConvKG achieves the better mean rank(MR)and Hits@10 metrics on dataset WN18 RR,and the better MR on dataset FB15k-237.展开更多
With the continuous improvement of the e-commerce ecosystem and the rapid growth of e-commerce data, inthe context of the e-commerce ecosystem, consumers ask hundreds of millions of questions every day. In order toimp...With the continuous improvement of the e-commerce ecosystem and the rapid growth of e-commerce data, inthe context of the e-commerce ecosystem, consumers ask hundreds of millions of questions every day. In order toimprove the timeliness of customer service responses, many systems have begun to use customer service robotsto respond to consumer questions, but the current customer service robots tend to respond to specific questions.For many questions that lack background knowledge, they can generate only responses that are biased towardsgenerality and repetitiveness. To better promote the understanding of dialogue and generate more meaningfulresponses, this paper introduces knowledge information into the research of question answering system by usinga knowledge graph. The unique structured knowledge base of the knowledge graph is convenient for knowledgequery, can acquire knowledge faster, and improves the background information needed for answering questions. Toavoid the lack of information in the dialogue process, this paper proposes the Multi-hop Knowledge InformationEnhanced Dialogue-Graph Attention (MKIED-GA) model. The model first retrieves the problem subgraph directlyrelated to the input information from the entire knowledge base and then uses the graph neural network as theknowledge inference module on the subgraph to encode the subgraph. The graph attention mechanism is usedto determine the one-hop and two-hop entities that are more relevant to the problem to achieve the aggregationof highly relevant neighbor information. This further enriches the semantic information to provide a betterunderstanding of the meaning of the input question and generate appropriate response information. In the processof generating a response, a multi-attention flow mechanism is used to focus on different information to promotethe generation of better responses. Experiments have proved that the model presented in this article can generatemore meaningful responses than other models.展开更多
Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting charac...Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting characteristic parameters describing HBF in multiple dimensions and multiple scales;(2)showing the characteristic parameter-related entities,relationships,and attributes as vectors via graph embedding technique;(3)intelligently identifying HBF;(4)seamlessly integrating expertise into the intelligent computing to establish the assessment system and ranking algorithm for potential pay recommendation.Taking 547 wells encountered the low porosity and low permeability Chang 6 Member of Triassic in the Jiyuan Block of Ordos Basin,NW China as objects,80%of the wells were randomly selected as the training dataset and the remainder as the validation dataset.The KPNFE prediction results on the validation dataset had a coincidence rate of 94.43%with the expert interpretation results and a coincidence rate of 84.38%for all the oil testing layers,which is 13 percentage points higher in accuracy and over 100 times faster than the primary conventional interpretation.In addition,a number of potential pays likely to produce industrial oil were recommended.The KPNFE model effectively inherits,carries forward and improves the expert knowledge,nicely solving the robustness problem in HBF identification.The KPNFE,with good interpretability and high accuracy of computation results,is a powerful technical means for efficient and high-quality well logging re-evaluation of old wells in mature oilfields.展开更多
Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Availabl...Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Available knowledge tracing models have the problem that the assessments are not directly used in the predictions.To make full use of the assessments during predictions,a novel model,named deep knowledge tracing embedding neural network(DKTENN),is proposed in this work.DKTENN is a synthesis of deep knowledge tracing(DKT)and knowledge graph embedding(KGE).DKT utilizes sophisticated long short-term memory(LSTM)to assess the users and track the mastery of skills according to the users'interaction sequences with skill-level tags,and KGE is applied to predict the probability on the basis of both the embedded problems and DKT's assessments.DKTENN outperforms performance factors analysis and the other knowledge tracing models based on deep learning in the experiments.展开更多
Using knowledge graphs to assist deep learning models in making recommendation decisions has recently been proven to effectively improve the model′s interpretability and accuracy.This paper introduces an end-to-end d...Using knowledge graphs to assist deep learning models in making recommendation decisions has recently been proven to effectively improve the model′s interpretability and accuracy.This paper introduces an end-to-end deep learning model,named representation-enhanced knowledge graph convolutional networks(RKGCN),which dynamically analyses each user′s preferences and makes a recommendation of suitable items.It combines knowledge graphs on both the item side and user side to enrich their representations to maximize the utilization of the abundant information in knowledge graphs.RKGCN is able to offer more personalized and relevant recommendations in three different scenarios.The experimental results show the superior effectiveness of our model over 5 baseline models on three real-world datasets including movies,books,and music.展开更多
Nowadays,ensuring thequality of networkserviceshas become increasingly vital.Experts are turning toknowledge graph technology,with a significant emphasis on entity extraction in the identification of device configurat...Nowadays,ensuring thequality of networkserviceshas become increasingly vital.Experts are turning toknowledge graph technology,with a significant emphasis on entity extraction in the identification of device configurations.This research paper presents a novel entity extraction method that leverages a combination of active learning and attention mechanisms.Initially,an improved active learning approach is employed to select the most valuable unlabeled samples,which are subsequently submitted for expert labeling.This approach successfully addresses the problems of isolated points and sample redundancy within the network configuration sample set.Then the labeled samples are utilized to train the model for network configuration entity extraction.Furthermore,the multi-head self-attention of the transformer model is enhanced by introducing the Adaptive Weighting method based on the Laplace mixture distribution.This enhancement enables the transformer model to dynamically adapt its focus to words in various positions,displaying exceptional adaptability to abnormal data and further elevating the accuracy of the proposed model.Through comparisons with Random Sampling(RANDOM),Maximum Normalized Log-Probability(MNLP),Least Confidence(LC),Token Entrop(TE),and Entropy Query by Bagging(EQB),the proposed method,Entropy Query by Bagging and Maximum Influence Active Learning(EQBMIAL),achieves comparable performance with only 40% of the samples on both datasets,while other algorithms require 50% of the samples.Furthermore,the entity extraction algorithm with the Adaptive Weighted Multi-head Attention mechanism(AW-MHA)is compared with BILSTM-CRF,Mutil_Attention-Bilstm-Crf,Deep_Neural_Model_NER and BERT_Transformer,achieving precision rates of 75.98% and 98.32% on the two datasets,respectively.Statistical tests demonstrate the statistical significance and effectiveness of the proposed algorithms in this paper.展开更多
In the link prediction task of knowledge graph completion,Graph Neural Network(GNN)-based knowledge graph completion models have been shown by previous studies to produce large improvements in prediction results.Howev...In the link prediction task of knowledge graph completion,Graph Neural Network(GNN)-based knowledge graph completion models have been shown by previous studies to produce large improvements in prediction results.However,many of the previous efforts were limited to aggregating the information given by neighboring nodes and did not take advantage of the information provided by the edges represented by relations.To address the problem,Coupling Relation Strength with Graph Convolutional Networks(RS-GCN)is proposed,which is a model with an encoder-decoder framework to realize the embedding of entities and relations in the vector space.On the encoder side,RS-GCN captures graph structure and neighborhood information while aggregating the information given by neighboring nodes.On the decoder side,RotatE is utilized to model and infer various relational patterns.The models are evaluated on standard FB15k,WN18,FB15k-237 and WN18RR datasets,and the experiments show that RS-GCN achieves better results than the current state-of-the-art classical models on the above knowledge graph datasets.展开更多
Commonsense question answering(CQA)requires understanding and reasoning over QA context and related commonsense knowledge,such as a structured Knowledge Graph(KG).Existing studies combine language models and graph neu...Commonsense question answering(CQA)requires understanding and reasoning over QA context and related commonsense knowledge,such as a structured Knowledge Graph(KG).Existing studies combine language models and graph neural networks to model inference.However,traditional knowledge graph are mostly concept-based,ignoring direct path evidence necessary for accurate reasoning.In this paper,we propose MRGNN(Meta-path Reasoning Graph Neural Network),a novel model that comprehensively captures sequential semantic information from concepts and paths.In MRGNN,meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously.We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets,showing the effectiveness of MRGNN.Also,we conduct further ablation experiments and explain the reasoning behavior through the case study.展开更多
Entity alignment(EA)is an important technique aiming to find the same real entity between two different source knowledge graphs(KGs).Current methods typically learn the embedding of entities for EA from the structure ...Entity alignment(EA)is an important technique aiming to find the same real entity between two different source knowledge graphs(KGs).Current methods typically learn the embedding of entities for EA from the structure of KGs for EA.Most EA models are designed for rich-resource languages,requiring sufficient resources such as a parallel corpus and pre-trained language models.However,low-resource language KGs have received less attention,and current models demonstrate poor performance on those low-resource KGs.Recently,researchers have fused relation information and attributes for entity representations to enhance the entity alignment performance,but the relation semantics are often ignored.To address these issues,we propose a novel Semantic-aware Graph Neural Network(SGNN)for entity alignment.First,we generate pseudo sentences according to the relation triples and produce representations using pre-trained models.Second,our approach explores semantic information from the connected relations by a graph neural network.Our model captures expanded feature information from KGs.Experimental results using three low-resource languages demonstrate that our proposed SGNN approach out performs better than state-of-the-art alignment methods on three proposed datasets and three public datasets.展开更多
基金supported in part by the Science and Technology Innovation 2030-“New Generation of Artificial Intelligence”Major Project(No.2021ZD0111000)Henan Provincial Science and Technology Research Project(No.232102211039).
文摘The growing prevalence of knowledge reasoning using knowledge graphs(KGs)has substantially improved the accuracy and efficiency of intelligent medical diagnosis.However,current models primarily integrate electronic medical records(EMRs)and KGs into the knowledge reasoning process,ignoring the differing significance of various types of knowledge in EMRs and the diverse data types present in the text.To better integrate EMR text information,we propose a novel intelligent diagnostic model named the Graph ATtention network incorporating Text representation in knowledge reasoning(GATiT),which comprises text representation,subgraph construction,knowledge reasoning,and diagnostic classification.In the text representation process,GATiT uses a pre-trained model to obtain text representations of the EMRs and additionally enhances embeddings by including chief complaint information and numerical information in the input.In the subgraph construction process,GATiT constructs text subgraphs and disease subgraphs from the KG,utilizing EMR text and the disease to be diagnosed.To differentiate the varying importance of nodes within the subgraphs features such as node categories,relevance scores,and other relevant factors are introduced into the text subgraph.Themessage-passing strategy and attention weight calculation of the graph attention network are adjusted to learn these features in the knowledge reasoning process.Finally,in the diagnostic classification process,the interactive attention-based fusion method integrates the results of knowledge reasoning with text representations to produce the final diagnosis results.Experimental results on multi-label and single-label EMR datasets demonstrate the model’s superiority over several state-of-theart methods.
基金supported in part by the Beijing Natural Science Foundation under Grants L211020 and M21032in part by the National Natural Science Foundation of China under Grants U1836106 and 62271045in part by the Scientific and Technological Innovation Foundation of Foshan under Grants BK21BF001 and BK20BF010。
文摘Knowledge graph(KG)serves as a specialized semantic network that encapsulates intricate relationships among real-world entities within a structured framework.This framework facilitates a transformation in information retrieval,transitioning it from mere string matching to far more sophisticated entity matching.In this transformative process,the advancement of artificial intelligence and intelligent information services is invigorated.Meanwhile,the role ofmachine learningmethod in the construction of KG is important,and these techniques have already achieved initial success.This article embarks on a comprehensive journey through the last strides in the field of KG via machine learning.With a profound amalgamation of cutting-edge research in machine learning,this article undertakes a systematical exploration of KG construction methods in three distinct phases:entity learning,ontology learning,and knowledge reasoning.Especially,a meticulous dissection of machine learningdriven algorithms is conducted,spotlighting their contributions to critical facets such as entity extraction,relation extraction,entity linking,and link prediction.Moreover,this article also provides an analysis of the unresolved challenges and emerging trajectories that beckon within the expansive application of machine learning-fueled,large-scale KG construction.
基金supported in part by the Major Scientific and Technological Projects of CNPC under Grant ZD2019-183-006.
文摘Link prediction,also known as Knowledge Graph Completion(KGC),is the common task in Knowledge Graphs(KGs)to predict missing connections between entities.Most existing methods focus on designing shallow,scalable models,which have less expressive than deep,multi-layer models.Furthermore,most operations like addition,matrix multiplications or factorization are handcrafted based on a few known relation patterns in several wellknown datasets,such as FB15k,WN18,etc.However,due to the diversity and complex nature of real-world data distribution,it is inherently difficult to preset all latent patterns.To address this issue,we proposeKGE-ANS,a novel knowledge graph embedding framework for general link prediction tasks using automatic network search.KGEANS can learn a deep,multi-layer effective architecture to adapt to different datasets through neural architecture search.In addition,the general search spacewe designed is tailored forKGtasks.We performextensive experiments on benchmark datasets and the dataset constructed in this paper.The results show that our KGE-ANS outperforms several state-of-the-art methods,especially on these datasets with complex relation patterns.
基金supported by the Sichuan Science and Technology Program under Grants No.2022YFQ0052 and No.2021YFQ0009.
文摘At present,knowledge embedding methods are widely used in the field of knowledge graph(KG)reasoning,and have been successfully applied to those with large entities and relationships.However,in research and production environments,there are a large number of KGs with a small number of entities and relations,which are called sparse KGs.Limited by the performance of knowledge extraction methods or some other reasons(some common-sense information does not appear in the natural corpus),the relation between entities is often incomplete.To solve this problem,a method of the graph neural network and information enhancement is proposed.The improved method increases the mean reciprocal rank(MRR)and Hit@3 by 1.6%and 1.7%,respectively,when the sparsity of the FB15K-237 dataset is 10%.When the sparsity is 50%,the evaluation indexes MRR and Hit@10 are increased by 0.8%and 1.8%,respectively.
基金supported by the Natural Science Foundation of China (62372277)the Natural Science Foundation of Shandong Province (ZR2022MF257, ZR2022MF295)Humanities and Social Sciences Fund of the Ministry of Education (21YJC630157)。
文摘With the popularity of online learning in educational settings, knowledge tracing(KT) plays an increasingly significant role. The task of KT is to help students learn more effectively by predicting their next mastery of knowledge based on their historical exercise sequences. Nowadays, many related works have emerged in this field, such as Bayesian knowledge tracing and deep knowledge tracing methods. Despite the progress that has been made in KT, existing techniques still have the following limitations: 1) Previous studies address KT by only exploring the observational sparsity data distribution, and the counterfactual data distribution has been largely ignored. 2) Current works designed for KT only consider either the entity relationships between questions and concepts, or the relations between two concepts, and none of them investigates the relations among students, questions, and concepts, simultaneously, leading to inaccurate student modeling. To address the above limitations,we propose a graph counterfactual augmentation method for knowledge tracing. Concretely, to consider the multiple relationships among different entities, we first uniform students, questions, and concepts in graphs, and then leverage a heterogeneous graph convolutional network to conduct representation learning.To model the counterfactual world, we conduct counterfactual transformations on students’ learning graphs by changing the corresponding treatments and then exploit the counterfactual outcomes in a contrastive learning framework. We conduct extensive experiments on three real-world datasets, and the experimental results demonstrate the superiority of our proposed Graph CA method compared with several state-of-the-art baselines.
文摘A knowledge graph consists of a set of interconnected typed entities and their attributes,which shows a better performance to organize,manage and understand knowledge.However,because knowledge graphs contain a lot of knowledge triples,it is difficult to directly display to researchers.Semantic Link Network is an attempt,and it can deal with the construction,representation and reasoning of semantics naturally.Based on the Semantic Link Network,this paper explores the representation and construction of knowledge graph,and develops an academic knowledge graph prototype system to realize the representation,construction and visualization of knowledge graph.
文摘Objective To study the research status,research hotspots and development trends in the field of real-world data(RWD)through social network analysis and knowledge graph analysis.Methods RWD of the past 10 years were retrieved,and literature metrological analysis was made by using UCINET and CiteSpace from CNKI.Results and Conclusion The frequency and centrality of related keywords such as real-world study,hospital information system(HIS),drug combination,data mining and TCM are high.The clusters labeled as clinical medication and RWD contain more keywords.In recent 4 years,there are more articles involving the keywords of data specification,data authenticity,data security and information security.Among them,compound Kushen injection,HIS database and RWD are the top three keywords.It is a long-term research hotspot for Chinese and western medicine to use HIS to study clinical medication,clinical characteristics,diseases and injections.Besides,the research of RWD database has changed from construction to standardized collection and governance,which can make RWD effective.Data authenticity,data security and information security will become the new hotspots in the research of RWD.
基金supported in part by the National Natural Science Foundation of China(61621003,62073079,62088101,12025107,11871463,11688101)。
文摘Along with the development of information technologies such as mobile Internet,information acquisition technology,cloud computing and big data technology,the traditional knowledge engineering and knowledge-based software engineering have undergone fundamental changes where the network plays an increasingly important role.Within this context,it is required to develop new methodologies as well as technical tools for network-based knowledge representation,knowledge services and knowledge engineering.Obviously,the term“network”has different meanings in different scenarios.Meanwhile,some breakthroughs in several bottleneck problems of complex networks promote the developments of the new methodologies and technical tools for network-based knowledge representation,knowledge services and knowledge engineering.This paper first reviews some recent advances on complex networks,and then,in conjunction with knowledge graph,proposes a framework of networked knowledge which models knowledge and its relationships with the perspective of complex networks.For the unique advantages of deep learning in acquiring and processing knowledge,this paper reviews its development and emphasizes the role that it played in the development of knowledge engineering.Finally,some challenges and further trends are discussed.
基金supported by the NationalKeyR&DProgramof China underGrant No.2018YFA0701604.
文摘Due to the many types of distributed denial-of-service attacks(DDoS)attacks and the large amount of data generated,it becomes a chal-lenge to manage and apply the malicious behavior knowledge generated by DDoS attacks.We propose a malicious behavior knowledge base framework for DDoS attacks,which completes the construction and application of a multi-domain malicious behavior knowledge base.First,we collected mali-cious behavior traffic generated by five mainstream DDoS attacks.At the same time,we completed the knowledge collection mechanism through data pre-processing and dataset design.Then,we designed a malicious behavior category graph and malicious behavior structure graph for the characteristic information and spatial structure of DDoS attacks and completed the knowl-edge learning mechanism using a graph neural network model.To protect the data privacy of multiple multi-domain malicious behavior knowledge bases,we implement the knowledge-sharing mechanism based on federated learning.Finally,we store the constructed knowledge graphs,graph neural network model,and Federated model into the malicious behavior knowledge base to complete the knowledge management mechanism.The experimental results show that our proposed system architecture can effectively construct and apply the malicious behavior knowledge base,and the detection capability of multiple DDoS attacks occurring in the network reaches above 0.95,while there exists a certain anti-interference capability for data poisoning cases.
基金Supported by the National Natural Science Foundation of China(No.61876144).
文摘Knowledge graph(KG) link prediction aims to address the problem of missing multiple valid triples in KGs. Existing approaches either struggle to efficiently model the message passing process of multi-hop paths or lack transparency of model prediction principles. In this paper,a new graph convolutional network path semantic-aware graph convolution network(PSGCN) is proposed to achieve modeling the semantic information of multi-hop paths. PSGCN first uses a random walk strategy to obtain all-hop paths in KGs,then captures the semantics of the paths by Word2Sec and long shortterm memory(LSTM) models,and finally converts them into a potential representation for the graph convolution network(GCN) messaging process. PSGCN combines path-based inference methods and graph neural networks to achieve better interpretability and scalability. In addition,to ensure the robustness of the model,the value of the path thresholdKis experimented on the FB15K-237 and WN18RR datasets,and the final results prove the effectiveness of the model.
基金Supported by the National Natural Science Foundation of China(No.61876144)。
文摘To solve the problem of missing many valid triples in knowledge graphs(KGs),a novel model based on a convolutional neural network(CNN)called ConvKG is proposed,which employs a joint learning strategy for knowledge graph completion(KGC).Related research work has shown the superiority of convolutional neural networks(CNNs)in extracting semantic features of triple embeddings.However,these researches use only one single-shaped filter and fail to extract semantic features of different granularity.To solve this problem,ConvKG exploits multi-shaped filters to co-convolute on the triple embeddings,joint learning semantic features of different granularity.Different shaped filters cover different sizes on the triple embeddings and capture pairwise interactions of different granularity among triple elements.Experimental results confirm the strength of joint learning,and compared with state-of-the-art CNN-based KGC models,ConvKG achieves the better mean rank(MR)and Hits@10 metrics on dataset WN18 RR,and the better MR on dataset FB15k-237.
基金Funder One,National Nature Science Foundation of China,Grant/Award No.61972357Funder Two,National Nature Science Foundation of China,Grant/Award No.61672337Funder Three,Guangxi Colleges and Universities Basic Ability Improvement Project of Young and Middle-Aged Teachers,Grant/Award No.2018KY0651.
文摘With the continuous improvement of the e-commerce ecosystem and the rapid growth of e-commerce data, inthe context of the e-commerce ecosystem, consumers ask hundreds of millions of questions every day. In order toimprove the timeliness of customer service responses, many systems have begun to use customer service robotsto respond to consumer questions, but the current customer service robots tend to respond to specific questions.For many questions that lack background knowledge, they can generate only responses that are biased towardsgenerality and repetitiveness. To better promote the understanding of dialogue and generate more meaningfulresponses, this paper introduces knowledge information into the research of question answering system by usinga knowledge graph. The unique structured knowledge base of the knowledge graph is convenient for knowledgequery, can acquire knowledge faster, and improves the background information needed for answering questions. Toavoid the lack of information in the dialogue process, this paper proposes the Multi-hop Knowledge InformationEnhanced Dialogue-Graph Attention (MKIED-GA) model. The model first retrieves the problem subgraph directlyrelated to the input information from the entire knowledge base and then uses the graph neural network as theknowledge inference module on the subgraph to encode the subgraph. The graph attention mechanism is usedto determine the one-hop and two-hop entities that are more relevant to the problem to achieve the aggregationof highly relevant neighbor information. This further enriches the semantic information to provide a betterunderstanding of the meaning of the input question and generate appropriate response information. In the processof generating a response, a multi-attention flow mechanism is used to focus on different information to promotethe generation of better responses. Experiments have proved that the model presented in this article can generatemore meaningful responses than other models.
基金Supported by the National Science and Technology Major Project(2016ZX05007-004)。
文摘Based on the well logging knowledge graph of hydrocarbon-bearing formation(HBF),a Knowledge-Powered Neural Network Formation Evaluation model(KPNFE)has been proposed.It has the following functions:(1)extracting characteristic parameters describing HBF in multiple dimensions and multiple scales;(2)showing the characteristic parameter-related entities,relationships,and attributes as vectors via graph embedding technique;(3)intelligently identifying HBF;(4)seamlessly integrating expertise into the intelligent computing to establish the assessment system and ranking algorithm for potential pay recommendation.Taking 547 wells encountered the low porosity and low permeability Chang 6 Member of Triassic in the Jiyuan Block of Ordos Basin,NW China as objects,80%of the wells were randomly selected as the training dataset and the remainder as the validation dataset.The KPNFE prediction results on the validation dataset had a coincidence rate of 94.43%with the expert interpretation results and a coincidence rate of 84.38%for all the oil testing layers,which is 13 percentage points higher in accuracy and over 100 times faster than the primary conventional interpretation.In addition,a number of potential pays likely to produce industrial oil were recommended.The KPNFE model effectively inherits,carries forward and improves the expert knowledge,nicely solving the robustness problem in HBF identification.The KPNFE,with good interpretability and high accuracy of computation results,is a powerful technical means for efficient and high-quality well logging re-evaluation of old wells in mature oilfields.
文摘Knowledge tracing is the key component in online individualized learning,which is capable of assessing the users'mastery of skills and predicting the probability that the users can solve specific problems.Available knowledge tracing models have the problem that the assessments are not directly used in the predictions.To make full use of the assessments during predictions,a novel model,named deep knowledge tracing embedding neural network(DKTENN),is proposed in this work.DKTENN is a synthesis of deep knowledge tracing(DKT)and knowledge graph embedding(KGE).DKT utilizes sophisticated long short-term memory(LSTM)to assess the users and track the mastery of skills according to the users'interaction sequences with skill-level tags,and KGE is applied to predict the probability on the basis of both the embedded problems and DKT's assessments.DKTENN outperforms performance factors analysis and the other knowledge tracing models based on deep learning in the experiments.
文摘Using knowledge graphs to assist deep learning models in making recommendation decisions has recently been proven to effectively improve the model′s interpretability and accuracy.This paper introduces an end-to-end deep learning model,named representation-enhanced knowledge graph convolutional networks(RKGCN),which dynamically analyses each user′s preferences and makes a recommendation of suitable items.It combines knowledge graphs on both the item side and user side to enrich their representations to maximize the utilization of the abundant information in knowledge graphs.RKGCN is able to offer more personalized and relevant recommendations in three different scenarios.The experimental results show the superior effectiveness of our model over 5 baseline models on three real-world datasets including movies,books,and music.
基金supported by the National Key R&D Program of China(2019YFB2103202).
文摘Nowadays,ensuring thequality of networkserviceshas become increasingly vital.Experts are turning toknowledge graph technology,with a significant emphasis on entity extraction in the identification of device configurations.This research paper presents a novel entity extraction method that leverages a combination of active learning and attention mechanisms.Initially,an improved active learning approach is employed to select the most valuable unlabeled samples,which are subsequently submitted for expert labeling.This approach successfully addresses the problems of isolated points and sample redundancy within the network configuration sample set.Then the labeled samples are utilized to train the model for network configuration entity extraction.Furthermore,the multi-head self-attention of the transformer model is enhanced by introducing the Adaptive Weighting method based on the Laplace mixture distribution.This enhancement enables the transformer model to dynamically adapt its focus to words in various positions,displaying exceptional adaptability to abnormal data and further elevating the accuracy of the proposed model.Through comparisons with Random Sampling(RANDOM),Maximum Normalized Log-Probability(MNLP),Least Confidence(LC),Token Entrop(TE),and Entropy Query by Bagging(EQB),the proposed method,Entropy Query by Bagging and Maximum Influence Active Learning(EQBMIAL),achieves comparable performance with only 40% of the samples on both datasets,while other algorithms require 50% of the samples.Furthermore,the entity extraction algorithm with the Adaptive Weighted Multi-head Attention mechanism(AW-MHA)is compared with BILSTM-CRF,Mutil_Attention-Bilstm-Crf,Deep_Neural_Model_NER and BERT_Transformer,achieving precision rates of 75.98% and 98.32% on the two datasets,respectively.Statistical tests demonstrate the statistical significance and effectiveness of the proposed algorithms in this paper.
文摘In the link prediction task of knowledge graph completion,Graph Neural Network(GNN)-based knowledge graph completion models have been shown by previous studies to produce large improvements in prediction results.However,many of the previous efforts were limited to aggregating the information given by neighboring nodes and did not take advantage of the information provided by the edges represented by relations.To address the problem,Coupling Relation Strength with Graph Convolutional Networks(RS-GCN)is proposed,which is a model with an encoder-decoder framework to realize the embedding of entities and relations in the vector space.On the encoder side,RS-GCN captures graph structure and neighborhood information while aggregating the information given by neighboring nodes.On the decoder side,RotatE is utilized to model and infer various relational patterns.The models are evaluated on standard FB15k,WN18,FB15k-237 and WN18RR datasets,and the experiments show that RS-GCN achieves better results than the current state-of-the-art classical models on the above knowledge graph datasets.
基金supported by the Key Research and Development Program of Hubei Province(2020BAB017)the Scientific Research Center Program of National Language Commission(ZDI135-135)the Fundamental Research Funds for the Central Universities(KJ02502022-0155,CCNU22XJ037).
文摘Commonsense question answering(CQA)requires understanding and reasoning over QA context and related commonsense knowledge,such as a structured Knowledge Graph(KG).Existing studies combine language models and graph neural networks to model inference.However,traditional knowledge graph are mostly concept-based,ignoring direct path evidence necessary for accurate reasoning.In this paper,we propose MRGNN(Meta-path Reasoning Graph Neural Network),a novel model that comprehensively captures sequential semantic information from concepts and paths.In MRGNN,meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously.We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets,showing the effectiveness of MRGNN.Also,we conduct further ablation experiments and explain the reasoning behavior through the case study.
基金National Natural Science Foundation of China(Nos.U21B2027,61972186,61732005)Major Science and Technology Projects of Yunnan Province(Nos.202202AD080003,202203AA080004).
文摘Entity alignment(EA)is an important technique aiming to find the same real entity between two different source knowledge graphs(KGs).Current methods typically learn the embedding of entities for EA from the structure of KGs for EA.Most EA models are designed for rich-resource languages,requiring sufficient resources such as a parallel corpus and pre-trained language models.However,low-resource language KGs have received less attention,and current models demonstrate poor performance on those low-resource KGs.Recently,researchers have fused relation information and attributes for entity representations to enhance the entity alignment performance,but the relation semantics are often ignored.To address these issues,we propose a novel Semantic-aware Graph Neural Network(SGNN)for entity alignment.First,we generate pseudo sentences according to the relation triples and produce representations using pre-trained models.Second,our approach explores semantic information from the connected relations by a graph neural network.Our model captures expanded feature information from KGs.Experimental results using three low-resource languages demonstrate that our proposed SGNN approach out performs better than state-of-the-art alignment methods on three proposed datasets and three public datasets.