ALBERT with Knowledge Graph Encoder Utilizing Semantic Similarity for Commonsense Question Answering

Recently,pre-trained language representation models such as bidirec-tional encoder representations from transformers(BERT)have been performing well in commonsense question answering(CSQA).However,there is a problem that the models do not directly use explicit information of knowledge sources existing outside.To augment this,additional methods such as knowledge-aware graph network(KagNet)and multi-hop graph relation network(MHGRN)have been proposed.In this study,we propose to use the latest pre-trained language model a lite bidirectional encoder representations from transformers(ALBERT)with knowledge graph information extraction technique.We also propose to applying the novel method,schema graph expansion to recent language models.Then,we analyze the effect of applying knowledge graph-based knowledge extraction techniques to recent pre-trained language models and confirm that schema graph expansion is effective in some extent.Furthermore,we show that our proposed model can achieve better performance than existing KagNet and MHGRN models in CommonsenseQA dataset.

Operational requirements analysis method based on question answering of WEKG

The weapon and equipment operational requirement analysis(WEORA) is a necessary condition to win a future war,among which the acquisition of knowledge about weapons and equipment is a great challenge. The main challenge is that the existing weapons and equipment data fails to carry out structured knowledge representation, and knowledge navigation based on natural language cannot efficiently support the WEORA. To solve above problem, this research proposes a method based on question answering(QA) of weapons and equipment knowledge graph(WEKG) to construct and navigate the knowledge related to weapons and equipment in the WEORA. This method firstly constructs the WEKG, and builds a neutral network-based QA system over the WEKG by means of semantic parsing for knowledge navigation. Finally, the method is evaluated and a chatbot on the QA system is developed for the WEORA. Our proposed method has good performance in the accuracy and efficiency of searching target knowledge, and can well assist the WEORA.

Expert Knowledge-Based Apparel Recommendation Question and Answer System

Aiming at the lack of professional knowledge to guide apparel recommendation,an apparel recommendation method based on image design expert knowledge has been proposed.Then,apparel recommendation knowledge graphs have been created and a apparel recommendation question and answer(Q&A)system has been designed and implemented.The question templates in the apparel recommendation domain were defined,the task of recognizing the named entities of question sentences was completed by the Bi-directional encoder representations from transformer-Bi-directional long short-term memory-conditional random field(BERT-BiLSTM-CRF)model,and the question template with the highest matching degree to the user’s question was obtained by using term frequency-inverse document frequency(TF-IDF)algorithm.The corresponding cypher graph database query statement was generated to retrieve the knowledge graph for answers,and iFLYTEK’s voice application programming interface(API)was called to implement the Q&A.The experimental results have shown that the Q&A system has a high accuracy rate and application value in the field of apparel recommendations.

Survey of Knowledge Graph Approaches and Applications

With the advent of the era of big data,knowledge engineering has received extensive attention.How to extract useful knowledge from massive data is the key to big data analysis.Knowledge graph technology is an important part of artificial intelligence,which provides a method to extract structured knowledge from massive texts and images,and has broad application prospects.The knowledge base with semantic processing capability and open interconnection ability can be used to generate application value in intelligent information services such as intelligent search,intelligent question answering and personalized recommendation.Although knowledge graph has been applied to various systems,the basic theory and application technology still need further research.On the basis of comprehensively expounding the definition and architecture of knowledge graph,this paper reviews the key technologies of knowledge graph construction,including the research progress of four core technologies such as knowledge extraction technology,knowledge representation technology,knowledge fusion technology and knowledge reasoning technology,as well as some typical applications.Finally,the future development direction and challenges of the knowledge graph are prospected.

Meta-path reasoning of knowledge graph for commonsense question answering

Commonsense question answering(CQA)requires understanding and reasoning over QA context and related commonsense knowledge,such as a structured Knowledge Graph(KG).Existing studies combine language models and graph neural networks to model inference.However,traditional knowledge graph are mostly concept-based,ignoring direct path evidence necessary for accurate reasoning.In this paper,we propose MRGNN(Meta-path Reasoning Graph Neural Network),a novel model that comprehensively captures sequential semantic information from concepts and paths.In MRGNN,meta-paths are introduced as direct inference evidence and an original graph neural network is adopted to aggregate features from both concepts and paths simultaneously.We conduct sufficient experiments on the CommonsenceQA and OpenBookQA datasets,showing the effectiveness of MRGNN.Also,we conduct further ablation experiments and explain the reasoning behavior through the case study.

COKG-QA: Multi-hop Question Answering over COVID-19 Knowledge Graphs

COVID-19 evolves rapidly and an enormous number of people worldwide desire instant access to COVID-19 information such as the overview, clinic knowledge, vaccine, prevention measures, and COVID-19 mutation. Question answering(QA) has become the mainstream interaction way for users to consume the ever-growing information by posing natural language questions. Therefore, it is urgent and necessary to develop a QA system to offer consulting services all the time to relieve the stress of health services. In particular, people increasingly pay more attention to complex multi-hop questions rather than simple ones during the lasting pandemic, but the existing COVID-19 QA systems fail to meet their complex information needs. In this paper, we introduce a novel multi-hop QA system called COKG-QA, which reasons over multiple relations over large-scale COVID-19 Knowledge Graphs to return answers given a question. In the field of question answering over knowledge graph, current methods usually represent entities and schemas based on some knowledge embedding models and represent questions using pre-trained models. While it is convenient to represent different knowledge(i.e., entities and questions) based on specified embeddings, an issue raises that these separate representations come from heterogeneous vector spaces. We align question embeddings with knowledge embeddings in a common semantic space by a simple but effective embedding projection mechanism. Furthermore, we propose combining entity embeddings with their corresponding schema embeddings which served as important prior knowledge, to help search for the correct answer entity of specified types. In addition, we derive a large multi-hop Chinese COVID-19 dataset(called COKG-DATA for remembering) for COKG-QA based on the linked knowledge graph Open KG-COVID-19 launched by Open KG1, including comprehensive and representative information about COVID-19. COKG-QA achieves quite competitive performance in the 1-hop and 2-hop data while obtaining the best result with significant improvements in the 3-hop. And it is more efficient to be used in the QA system for users. Moreover, the user study shows that the system not only provides accurate and interpretable answers but also is easy to use and comes with smart tips and suggestions.

基于知识图谱多跳推理的中文矿物知识问答方法与系统

已有相关矿物数据库用于存储和查询相关矿物知识,常用的搜索引擎也可以对矿物知识进行查询,但无法回答用自然语言进行提问的矿物问题,查询返回的答案需要进一步筛选。亦有基于知识图谱进行矿物知识问答的相关研究,但只能回答涉及知识图谱中一个三元组的简单问题,无法回答涉及多个三元组的多跳复杂问题。为此,本文提出基于知识图谱多跳推理的矿物复杂知识问答方法,采用ComplEx模型将矿物实体、关系和问句表示为复数向量,以更好地获取相互之间的语义及推理关系。输入矿物问句后,通过Bert-LSTM-CRF获取其中心词,采用基于编辑距离及分词的方法获得中心词的候选实体集合,然后采用全连接网络确定最相关的实体作为推理起点,与矿物问句拼接后通过全连接网络获得当前跳的最相关关系。根据当前跳的起始实体及最相关关系,在矿物知识图谱中获得另一实体作为下一跳的推理起点,并将下一跳的问句更新为原问句,与当前跳最相关关系拼接,以将当前跳的推理信息带入到下一跳推理中,直到获得的最相关推理关系为预定义的结束标识符,推理结束,返回最后一跳的实体为答案,并给出推理路径。采用Python语言,在Tensorflow框架下实现了本文提出的矿物复杂知识问答并与相关模型进行对比,证明了本文方法的有效性。采用前后端分离架构,使用RESTful API、React、Ajax、echarts和Flask等框架和技术,开发了基于知识图谱多跳推理的矿物复杂知识问答系统,为矿物知识获取及相关地质研究提供了平台和工具。

反绎学习支持下的自动问答及其应用

自动问答技术可以为用户提供快速且准确的信息检索和问题解答服务。然而,目前常见方法生成的答案存在不准确和不完整的问题,以及实体识别和关系抽取效果不准确,且答案不够自然。为此,提出基于反绎学习的自动问答方法,使用基于知识图谱的问答推理优化基于生成的问答,进一步从整体的反绎学习框架角度来优化实体识别和关系抽取方法,并将所提方法应用于《数据结构》课程的学习。结果表明,基于反绎学习的自动问答方法,可以改进基于生成的问答和基于知识图谱的问答两者的不足,提高问答系统的准确性。

融入三维语义特征的常识推理问答方法

现有使用预训练语言模型和知识图谱的常识问答方法主要集中于构建知识图谱子图及跨模态信息结合的研究,忽略了知识图谱自身丰富的语义特征,且缺少对不同问答任务的知识图谱子图节点相关性的动态调整,导致预测准确率低。为解决以上问题,提出一种融入三维语义特征的常识推理问答方法。首先提出知识图谱节点的关系层级、实体层级、三元组层级三维语义特征量化指标;其次,通过注意力机制动态计算关系层级、实体层级、三元组层级三种维度的语义特征对不同实体节点间的重要性;最后,通过图神经网络进行多层聚合迭代嵌入三维语义特征,获得更多的外推知识表示,更新知识图谱子图节点表示,提升答案预测精度。与QA-GNN常识问答推理方法相比,所提方法在CommonsenseQA数据集上的验证集和测试集的准确率分别提高了1.70个百分点和0.74个百分点,在OpenBookQA数据集上使用AristoRoBERTa数据处理方法的准确率提高了1.13个百分点。实验结果表明,所提出的融入三维语义特征的常识推理问答方法能够有效提高常识问答任务准确率。

大语言模型融合知识图谱与向量检索的问答系统

随着深度学习的发展,大型神经网络在自然语言处理领域得到广泛应用。然而,基于大模型的问答系统存在幻觉、失效过期等问题,且难以捕捉实体之间的复杂关系,导致结果偏差。鉴于此,提出一种利用大语言模型微调构建知识图谱和向量检索的融合问答系统。系统通过微调大模型实现知识图谱构建与应用、多模型混合调用;结合知识图谱搜索和向量搜索实现检索结果优化。系统包括图查询模型微调、知识图谱抽取模型微调、知识图谱与向量数据库构建、融合检索与排序4个模块。图查询模型和知识图谱抽取模型分别用于生成图查询语句和抽取三元组知识;知识图谱存储在Neo4j中,文本向量存储在向量数据库中(postgres vector,PGVector)中;融合检索综合利用知识图谱和向量搜索结果。结果表明:在标准问答数据集(the stanford question answering dataset,SQuAD)上,融合检索方法的F1值为0.77,优于单一的知识图谱检索(0.73)和向量检索(0.74)方法。专家评估也表明,融合方法的结果最佳。该融合问答系统能充分发挥大模型、知识图谱和向量检索的优势,提高了问答的准确性和全面性。未来可在知识图谱更新、模型偏见减少和系统优化等方面开展进一步研究。

利用知识图谱的多跳可解释问答

基于知识图谱的多跳问答需要分析和理解自然语言问题并在知识图谱的实体和关系上经过多次推理获取答案,是自然语言处理的重要研究领域.现有的模型一般通过知识图谱与问题嵌入,利用神经网络推断答案;或使用一阶逻辑规则结合概率方法预测答案;前者缺乏可解释性,后者在复杂问题中性能欠佳.为解决上述问题,本文提出一种基于知识图谱的多跳可解释问答方法(MIQA),它通过在实体间的多次跳跃推理来获取答案.MIQA首先使用BERT预训练模型获取自然语言问题表征向量以及问题分词后的词向量矩阵,在每一跳中,结合问题向量提取问题当前时刻的特征向量,根据特征向量的分类结果计算下一跳的关系分数和实体分数,多次跳跃后,综合分数最高的实体被作为答案,而获取该答案所对应的路径为推理路径.该方法推理准确率高,同时具有明显的可解释性.在MetaQA、WebQuestionsSP、ComplexWebQuestions这3个数据集上,通过和其他8个知名算法相比较,仿真结果表明MIQA性能优异,达到了当前的SOTA.

一种引入核心实体关注度评估的KBQA算法

目前针对复杂语义和复杂句法的知识库问答(Knowledge Base Question Answering,KBQA)研究层出不穷,但它们多以已知问题的主题实体为前提,对问题中多意图和多实体重视不足,而问句中对核心实体的识别是理解自然语言的关键。针对此问题,提出了一种引入核心实体关注度的KBQA模型。该模型基于注意力机制及注意力增强技术,对识别到的实体引用(Mention)进行重要性评估,得到实体引用关注度,去除潜在干扰项,捕获用户提问的核心实体,解决了多实体、多意图问句的语义理解问题。此外,还将评估的结果作为重要权重引入后续的问答推理中。在英文MetaQA数据集、多实体问句MetaQA数据集、多实体问句HotpotQA数据集上,与KVMem,GraftNet,PullNet等模型进行了对比实验。结果表明,针对多实体问句,所提模型在Hits@n、准确率、召回率等评估指标上均取得了更好的实验效果。

大语言模型辅助下的增强现实装配方法

基于增强现实的装配引导系统将数字信息叠加到物理场景中,有效指导了复杂装配作业任务。然而装配环境中人与物理世界的隔阂仍然巨大,待融合到物理世界的信息需事先准备好,并且需要人工在装配过程中来触发。研究实时且无处不在的提示,成为当前增强现实环境下的复杂装配研究热点,本文提出了一种基于大语言模型(LLMs)辅助的增强现实装配方法,其核心是将LLMs作为装配过程中的另外一个大脑,提供无处不在的装配引导和工艺信息提示支持。首先,建立了LLMs辅助的增强现实装配方法体系,分析了体系的要素及关联关系。其次,面向LLMs环境,构建了匹配的工艺信息模型。随后,给出了基于LLMs的辅助引导装配方法和流程。最后,结合某线缆装配专业知识,研发了一个专业问答系统,实现了LLMs智能辅助引导,使装配合格率提升了15%,并通过多个案例验证了该方法的有效性。

融合GPT和知识图谱的洪涝应急决策智能问答系统研究

为提高生成式预训练语言大模型(generative pre-trained transformer, GPT)的应急管理信息分析能力,以实现洪涝灾害应急处置过程中的在线辅助决策,提出融合GPT和知识图谱的应急决策智能问答系统(KG-GPT)。改进GPT架构以识别问题中的关键信息,利用知识图谱推理应急领域知识并生成具有逻辑性的回答;结合洪涝灾害的实际应急决策问答数据集并编制演练脚本,使用自动评估和专家评估方法将本系统与GPT进行对比实验。研究结果表明:该系统成功融合应急领域知识图谱和GPT模型,能够深刻理解问题的背景信息并生成流畅回答;与GPT相比,该系统可为决策者提供更快速准确的在线辅助决策工具。研究结果可提升洪涝灾害应急信息分析和决策效率。

基于问题与关系嵌入空间对齐的知识图谱问答

为解决基于嵌入的知识图谱问答中,因采用不同的训练模型导致问题嵌入与知识图谱嵌入处于不同语义空间的问题,提出一种知识图谱问答模型。将关系转换为手工构造的自然语言问题,通过神经网络训练问题的嵌入表示与知识图谱的关系嵌入表示尽可能靠近;通过训练集中的问题对神经网络参数进行微调,使答案获得最高评分。在WebquestionSP数据集上的实验结果表明,相较于EmbedKGQA,所提方法的hits@1指标提高了10.5个百分点;在缺失50%三元组的情况下hits@1指标提高了9.9个百分点。

航空发动机润滑系统故障知识图谱构建及应用

由于航空发动机润滑系统结构功能复杂,基于现有的健康管理系统开展故障诊断存在可解释性不足及高度依赖专家经验的问题,提出一套面向航空发动机润滑系统的故障知识图谱构建方法。在结合专家知识设计润滑系统故障知识图谱本体概念的基础上,采用双向长短期记忆(BiLSTM)神经网络和条件随机场(CRF)等深度学习技术实现知识自主抽取,并基于余弦距离和Jaccard相关系数法进行多源异构故障知识的融合。同时,基于构建的润滑系统故障知识图谱,实现润滑系统故障知识智能问答和故障归因分析应用。结果表明:知识图谱技术能够实现对润滑系统故障先验知识利用及故障原因解释,在智能故障诊断领域具有良好应用前景。

面向地质图的知识图谱构建及智能问答应用

海量地质图件蕴含着丰富的地学基础知识及专家经验知识。地质图主要表达了通过区域地质调查、矿产地质调查所获取的地球表面的地质知识(如地层单元、岩体、断裂等)。如何快速地从矢量地质图件中抽取地质知识并形成知识服务是目前地学知识图谱及知识服务研究的前沿。由于传统的地质图知识抽取主要依赖人工方式进行综合分析,本文聚焦于矢量地质图件知识表达与抽取研究,提出了一种地质图知识表达框架,提取地质图中所包含的地质实体及关系,将地质图信息以知识图谱的形式表达,并开展了基于地质矢量知识图谱的智能问答应用。最后以江西省于都县银坑幅矢量数据集为例开展实验验证分析,结果表明,本文方法能够较为全面地获取地质图中各个地质对象的信息,提高了地质图语义表达的效果,同时也可以提高地质学习人员对地质图的理解和认识,让计算机能够大规模获取地质图的知识内容。

基于知识图谱的数字文物平台的研究与实现

在数字化时代背景下,文物的保护与传承面临着新的挑战与机遇,本文探讨并实现一种基于知识图谱的数字文物平台,以促进文物信息的整合、展示与利用。通过采集的丰富的文物数据,包括名称、简介、图片以及分类标签,以知识图谱的形式将这些数据进行组织与整合,存储于图数据库中,从而实现对文物数据的高效管理。其次,借助前端技术实现知识图谱的直观的可视化展示,用户可以通过交互式界面深入了解文物之间的关联、历史背景及分类情况。同时,基于知识图谱在平台中引入智能问答功能,为用户提供全面、准确的解答。本研究不仅为数字文物平台的构建提供了一种新的方法和技术支持,同时也为文物领域的数字化转型与应用提供了有益的参考与借鉴。

基于知识图谱的问答系统研究

随着信息时代的快速发展,问答系统作为知识获取的重要工具,其研究和应用价值日益凸显。传统的问答系统主要依赖于关键词匹配或预先定义模板,难以处理复杂、具有深度的问题,而基于知识图谱的问答系统应用先进的语言系统进行问题解答,用户能高效、准确地获取所需信息。文章将深入探讨知识图谱的构建、表示与更新,以及基于知识图谱的问答系统的设计、实现与应用,通过案例分析,揭示其面临的挑战与未来发展方向,并为问答系统的进一步发展提供有益的参考和启示,希望能够给人们获取知识带来更为便捷和智能的体验。

基于层次结构图的多跳知识图谱问答模型

知识图谱问答(KBQA)旨在理解用户的自然语言问句,在结构化的知识图谱中通过检索、推理等手段来获取答案实体。近年来,多跳KBQA备受关注,然而,复杂问句中通常存在多个关系意图,已有KBQA方法大多忽视了推理关系链的关系顺序问题。为此,提出一种基于层次结构图的多跳知识图谱问答模型(HSG-KBQA),建模自然语言问句的关系层次顺序,指导模型在每个推理步选择合理的关系意图。设计一种层次结构图,显式地体现问句中关系的层次距离,利用LSTM-BiGCN编码层将词语间的依存信息编码到问句中;提出虚拟节点的概念,利用图池化技术过滤不重要的节点,学习推理过程中知识图谱的状态;设计基于注意力机制和层次权重的解码器来优化指令生成,使推理指令更匹配问句中的关系链顺序。实验结果表明,HSG-KBQA在WebQuestionsSP数据集上取得了71.3%的Hits@1分数,在PathQuestions数据集上取得了97.3%(PQ-2H)和89.7%(PQ-3H)的Hits@1分数,均优于对照基准模型,表明HSG-KBQA模型在KBQA任务中具有更好的性能。