Enhancing Relational Triple Extraction in Specific Domains:Semantic Enhancement and Synergy of Large Language Models and Small Pre-Trained Language Models

In the process of constructing domain-specific knowledge graphs,the task of relational triple extraction plays a critical role in transforming unstructured text into structured information.Existing relational triple extraction models facemultiple challenges when processing domain-specific data,including insufficient utilization of semantic interaction information between entities and relations,difficulties in handling challenging samples,and the scarcity of domain-specific datasets.To address these issues,our study introduces three innovative components:Relation semantic enhancement,data augmentation,and a voting strategy,all designed to significantly improve the model’s performance in tackling domain-specific relational triple extraction tasks.We first propose an innovative attention interaction module.This method significantly enhances the semantic interaction capabilities between entities and relations by integrating semantic information fromrelation labels.Second,we propose a voting strategy that effectively combines the strengths of large languagemodels(LLMs)and fine-tuned small pre-trained language models(SLMs)to reevaluate challenging samples,thereby improving the model’s adaptability in specific domains.Additionally,we explore the use of LLMs for data augmentation,aiming to generate domain-specific datasets to alleviate the scarcity of domain data.Experiments conducted on three domain-specific datasets demonstrate that our model outperforms existing comparative models in several aspects,with F1 scores exceeding the State of the Art models by 2%,1.6%,and 0.6%,respectively,validating the effectiveness and generalizability of our approach.

Interaction between semantic and phonological processes in stuttering Evidence from the dual-task paradigm

Stuttering is a common neurological deficit and its underlying cognitive mechanisms are a matter of debate, with evidence suggesting abnormal modulation between speech encoding and other cognitive components. Previous studies have mainly used single task experiments to investigate the disturbance of language production. It is unclear whether there is abnormal interaction between the three language tasks （orthographic, phonological and semantic judgment） in stuttering patients. This study used dual tasks and manipulated the stimulus onset asynchrony （SOA） between tasks 1 and 2 and the nature of the second task, including orthographic, phonological, and semantic judgments. The results showed that the performance records of orthographic judgment, phonological judgment, and semantic judgment were significantly reduced between the patient and control groups with short SOA （P 〈 0.05）. However, different patterns of interaction between task 2 and SOA were observed across subject groups： subjects with stuttering were more strongly modulated by SOA when the second task was semantic judgment or phonological judgment （P 〈 0.05）, but not in the orthographic judgment experiment （P 〉 0.05）. These results indicated that the interaction mechanism between semantic processing and phonological encoding might be an underlying cause for stuttering.

ST-SIGMA:Spatio-temporal semantics and interaction graph aggregation for multi-agent perception and trajectory forecasting

Scene perception and trajectory forecasting are two fundamental challenges that are crucial to a safe and reliable autonomous driving(AD)system.However,most proposed methods aim at addressing one of the two challenges mentioned above with a single model.To tackle this dilemma,this paper proposes spatio-temporal semantics and interaction graph aggregation for multi-agent perception and trajectory forecasting(STSIGMA),an efficient end-to-end method to jointly and accurately perceive the AD environment and forecast the trajectories of the surrounding traffic agents within a unified framework.ST-SIGMA adopts a trident encoder-decoder architecture to learn scene semantics and agent interaction information on bird’s-eye view(BEV)maps simultaneously.Specifically,an iterative aggregation network is first employed as the scene semantic encoder(SSE)to learn diverse scene information.To preserve dynamic interactions of traffic agents,ST-SIGMA further exploits a spatio-temporal graph network as the graph interaction encoder.Meanwhile,a simple yet efficient feature fusion method to fuse semantic and interaction features into a unified feature space as the input to a novel hierarchical aggregation decoder for downstream prediction tasks is designed.Extensive experiments on the nuScenes data set have demonstrated that the proposed ST-SIGMA achieves significant improvements compared to the state-of-theart(SOTA)methods in terms of scene perception and trajectory forecasting,respectively.Therefore,the proposed approach outperforms SOTA in terms of model generalisation and robustness and is therefore more feasible for deployment in realworld AD scenarios.

A Moving Human Tracking Approach Based on Semantic Interaction

In order to deal with partical occlusion, a semantic interaction based moving human tracking approach is put forward. Firstly human is modeled as moving blobs which are described as blob descriptions. Then moving blobs are updated and verified by projecting these descriptions. The approach exploits improved fast gauss transform and chooses source and target samples to reduce compute cost. Multi-moving human can be tracked simply and part occlusion can be done well.

Interpreting and Extracting Open Knowledge for Human-Robot Interaction

A more natural way for non-expert users to express their tasks in an open-ended set is to use natural language. In this case,a human-centered intelligent agent/robot is required to be able to understand and generate plans for these naturally expressed tasks. For this purpose, it is a good way to enhance intelligent robot's abilities by utilizing open knowledge extracted from the web, instead of hand-coded knowledge. A key challenge of utilizing open knowledge lies in the semantic interpretation of the open knowledge organized in multiple modes, which can be unstructured or semi-structured, before one can use it.Previous approaches used a limited lexicon to employ combinatory categorial grammar(CCG) as the underlying formalism for semantic parsing over sentences. Here, we propose a more effective learning method to interpret semi-structured user instructions. Moreover, we present a new heuristic method to recover missing semantic information from the context of an instruction. Experiments showed that the proposed approach renders significant performance improvement compared to the baseline methods and the recovering method is promising.

BERT for Conversational Question Answering Systems Using Semantic Similarity Estimation

Most of the questions from users lack the context needed to thoroughly understand the problemat hand,thus making the questions impossible to answer.Semantic Similarity Estimation is based on relating user’s questions to the context from previous Conversational Search Systems(CSS)to provide answers without requesting the user’s context.It imposes constraints on the time needed to produce an answer for the user.The proposed model enables the use of contextual data associated with previous Conversational Searches(CS).While receiving a question in a new conversational search,the model determines the question that refers tomore pastCS.Themodel then infers past contextual data related to the given question and predicts an answer based on the context inferred without engaging in multi-turn interactions or requesting additional data from the user for context.This model shows the ability to use the limited information in user queries for best context inferences based on Closed-Domain-based CS and Bidirectional Encoder Representations from Transformers for textual representations.

Optimized Deep Learning Model for Fire Semantic Segmentation

Recent convolutional neural networks(CNNs)based deep learning has significantly promoted fire detection.Existing fire detection methods can efficiently recognize and locate the fire.However,the accurate flame boundary and shape information is hard to obtain by them,which makes it difficult to conduct automated fire region analysis,prediction,and early warning.To this end,we propose a fire semantic segmentation method based on Global Position Guidance(GPG)and Multi-path explicit Edge information Interaction(MEI).Specifically,to solve the problem of local segmentation errors in low-level feature space,a top-down global position guidance module is used to restrain the offset of low-level features.Besides,an MEI module is proposed to explicitly extract and utilize the edge information to refine the coarse fire segmentation results.We compare the proposed method with existing advanced semantic segmentation and salient object detection methods.Experimental results demonstrate that the proposed method achieves 94.1%,93.6%,94.6%,95.3%,and 95.9%Intersection over Union(IoU)on five test sets respectively which outperforms the suboptimal method by a large margin.In addition,in terms of accuracy,our approach also achieves the best score.

A Q-based integrating interaction framework system for multi-agent coordination

Interaction is one of the crucial features of multi-agent systems, in which there are two kinds of interaction: agent-to-agent and human-to-agent. In order to unify the two kinds of interaction while designing multi-agent systems, this paper introduces Qlanguage-a scenario description language for designing interaction among agents and humans. Based on Q,we propose an integrating interaction framework system for multi-agent coordination, in which Qscenarios are used to uniformly describe both kinds of interactions. Being in accordance to the characteristics of Qlanguage, the Q-basedframework makes the interaction process open and easily understood by the users. Additionally, it makes specific applications of multi-agent systems easy to be established by application designers. By applying agent negotiation in agent-mediated e-commerce and agent cooperation in interoperable information query on the Semantic Web, we illustrate how the presented framework for multi-agent coordination is implemented in concrete applications. At the same time, these two different applications also demonstrate usability of the presented framework and verify validity of Qlanguage.

深度协同感知的因子分解机

因子分解机(Factorization Machines,FM)在不同的输入实例中对每个特征产生单一的固定表示,忽略了特征的多语义特性,限制了点击率(Click-through Rate,CTR)预估模型的表示和预测能力.针对这一问题,提出一种深度协同感知的因子分解机模型,引入多语义交互感知网络和三重输入感知网络,通过多语义的特征域交互并融合不同层级的特征交互信息学习不同样本的感知因子,从而获得更加准确的特征表示.通过模型对比实验和消融实验表明:该模型可以有效提升点击率预测的准确性.

跨模态语义时空动态交互情感分析研究

针对传统情感分析中存在的模态间交互性差、时空特征融合度低的问题,建立了一种跨模态的语义时空动态交互网络。通过引入双向长短期记忆网络挖掘各模态的时间序列特征,加入自注意力机制强化模态内特征的权重赋值,将自动筛选出的特征矩阵送入图卷积神经网络进行语义交互。然后以时间戳为基础进行特征聚合,计算聚合层的相关系数,获得融合后的联合特征,实现跨模态空间交互,最终完成情感极性的分类与预测。使用公开数据集对所提出的模型进行评估验证,实验结果表明,多模态时间序列提取和跨模态语义空间交互机制可以实现模态内和模态间特征的全动态融合,有效地提升了情感分类的准确率和F1值,在CMU-MOSEI数据集上分别提高了1.7%~13.5%和2.1%~14.0%,表现出良好的健壮性和先进性。

基于多通道的语义信息融合交互方法

提出一种基于多通道的语义信息融合交互方法,使用不同的网络结构来提取原始语音信息、图像信息以及行为信息的语义特征,通过隐马尔可夫模型加强不同特征之间的交互,使用注意力机制建立语义信息融合,捕获了深层语义特征。在IEMOCAP数据集上验证了所提方法的有效性。

语义增强与高阶强交互的SAR图像舰船检测

合成孔径雷达(synthetic aperture radar,SAR)图像背景信息复杂、舰船目标边缘模糊,且多为容易丢失的小尺度舰船目标。针对上述问题,提出语义增强与高阶强交互的SAR图像舰船检测。该方法利用部分卷积与非对称卷积构建部分非对称卷积聚合网络,在减少计算复杂度、轻量化主干网络的同时,更好地捕捉多尺度舰船特征,同时在上采样部分引入双层路由注意力,增强对图像上下文信息的利用。另外,通过递归的方式进行特征提取,可以较好解决区域内信息交互的问题,实现不同级别特征之间的高阶交互建模,提升模型检测能力。在公开的HRSID遥感数据集上进行实验的结果表明,该方法的检测精度达到91.23%,相比原模型提升5.13%,准确率与召回率分别提升2.41%和7.16%,与主流算法相比具有较好的检测效果。

外部注意力增强语义交互的阅读理解模型

针对传统抽取式阅读理解模型未充分考虑问答样本之间潜在相关性的问题,通过RoBERTa对问题与段落进行编码,利用外部注意力Exatt增强语义交互层特征获取能力,提出外部注意力增强语义交互的阅读理解模型,捕获问题与段落中蕴涵的语义特征和不同问答样本之间的潜在相关性。实验结果表明,在CMRC2018和构建的电力安规问答数据集上,在评价指标EM和F1两方面,该方法较基线模型分别最高提高了0.737%和2.556%。

结合三维交互注意力与语义聚合的表情识别

针对传统卷积网络难以有效整合不同阶段人脸面部表情的特征、存在特征表征瓶颈以及无法高效利用上下文语义等问题,提出了一种结合三维交互注意力与语义聚合的面部表情识别方法。在秩扩展(ReXNet)网络的基础上对其进行优化,在消除表征瓶颈的情况下,融合上下文特征,使其更适配表情识别任务。为捕获判别性人脸表情细粒度特征,结合非本地块与跨维度信息交互理论构建了三维交互注意力。为充分利用表情的浅中层底层特征与高层语义特征,设计了语义聚合模块,将多级全局上下文特征与高级语义信息进行聚合,达到同一类别的表情语义相互增益、增强类内一致性的目的。实验表明,该方法在公开数据集RAF-DB、FERPlus和AffectNet-8上的准确率分别为88.89%、89.53%与62.22%,展现了该方法的先进性。

融合高斯过程的自支持小样本语义分割

针对小样本语义分割中同类别支持图像与查询图像存在外观差异较大的问题,提出融合高斯过程的自支持匹配小样本语义分割模型。提出的模型在自支持匹配小样本语义分割模型的基础上,首先融入高斯过程,对分布在深层特征空间上的复杂外观进行建模,捕获更多空间细节信息来表示数据分布;随后设计特征增强模块,在空间层对支持特征与查询特征进行信息交互,在通道层进行注意力加权,进一步增强相同类之间的全局相似性,捕获更多目标类别信息;最后利用Gram矩阵量化支持图像和查询图像之间外观差异的大小,从而融合原型匹配的结果,产生更准确的分割图像。实验结果表明:与现有方法相比,所提模型在更强的主干网络下具有较好的分割结果和更少的参数量,在5-shot的设定下,所提模型在PASCAL−5i数据集上平均交并比(mean Intersection over Union,mIoU)达到最优值,提升了0.4%;在COCO−20i数据集上的子集mIoU取得最优值,分别提升了2.2%和1.0%,表明该模型的有效性和先进性。

基于邻域关系感知图神经网络的DDI预测

研究药物的相互作用DDI有助于临床用药与新药研发。现有的研究技术没有充分考虑药物知识图谱中药物实体与其他药物、靶标和基因等实体的拓扑结构,以及实体之间不同关系的语义重要性。针对这些问题,提出基于邻域关系感知的图神经网络模型NRAGNN预测药物的相互作用。首先,使用图注意力学习不同关系边的权重与特征表示,强化药物实体的语义特征;然后,生成药物实体周围不同层的邻域表示,捕获药物实体的拓扑结构特征;最后,将2种药物特征表示向量进行逐元素相乘得到药物相互作用分数。实验预测结果表明,提出的NRAGNN模型在KEGG药物数据集上的ACC、AUPR、AUC-ROC和F1指标分别达到了0.8994,0.9444,0.9567和0.9023,优于当前的其他模型。

基于层级池化序列匹配的知识图谱复杂问答最优查询图选择方法

在处理知识图谱复杂问答任务时,传统的查询图语义解析方法需要在排序阶段对大量结构复杂的候选查询图进行语义编码,用以获得各自多维特征表示。然而,在编码过程中采用的全局最大或平均池化操作通常存在对代表性特征提取能力不足的问题。针对以上问题,提出一种基于层级池化序列匹配的最优查询图选择方法。在实现候选查询图的交互建模过程中,同时采用层级池化滑动窗口技术分层提取问句和查询图序列对的局部显著性特征与全局语义特征,使得到的特征向量更好地用于候选查询图的语义匹配打分。所提方法在两个流行的复杂问答数据集MetaQA和WebQuestionsSP上开展广泛实验。实验结果表明:引入层级池化操作能够有效提取复杂查询图序列的代表性语义特征,增强原有排序模型的交互编码能力,有助于进一步提升知识图谱复杂问答系统的性能。

问答模式下结合属性语义的实体属性抽取研究

实体属性抽取任务中常面临属性标签过多时模型存在爆炸风险的问题,且目前大多数属性抽取模型对文本均分配一致的注意力因子,未将上下文的变化考虑在内。为解决上述问题,提出一种基于问答模式的结合属性语义的实体属性抽取方法。该方法的要点在于,将文本看作上下文,把属性视为查询,从上下文中提取到的答案等同于期望的属性值。文中对文本和属性的语义表示进行建模,并提出一个动态注意力机制用于捕捉二者间的语义交互、实现信息融合,同时自适应地控制属性信息融入文本向量的程度。为了验证该方法的有效性,将模型与目前广泛应用的BiLSTM模型、BiLSTM-CRF模型、OpenTag模型和Open Tagging模型在包含大量属性标签的数据集AE-110K、AE-650K上进行对比实验,结果表明,模型在结合属性语义信息且采用动态Attention的条件下,其预测准确度、召回率和F1值更高。