A Novel Graph Structure Learning Based Semi-Supervised Framework for Anomaly Identification in Fluctuating IoT Environment

With the rapid development of Internet of Things(IoT)technology,IoT systems have been widely applied in health-care,transportation,home,and other fields.However,with the continuous expansion of the scale and increasing complexity of IoT systems,the stability and security issues of IoT systems have become increasingly prominent.Thus,it is crucial to detect anomalies in the collected IoT time series from various sensors.Recently,deep learning models have been leveraged for IoT anomaly detection.However,owing to the challenges associated with data labeling,most IoT anomaly detection methods resort to unsupervised learning techniques.Nevertheless,the absence of accurate abnormal information in unsupervised learning methods limits their performance.To address these problems,we propose AS-GCN-MTM,an adaptive structural Graph Convolutional Networks(GCN)-based framework using a mean-teacher mechanism(AS-GCN-MTM)for anomaly identification.It performs better than unsupervised methods using only a small amount of labeled data.Mean Teachers is an effective semi-supervised learning method that utilizes unlabeled data for training to improve the generalization ability and performance of the model.However,the dependencies between data are often unknown in time series data.To solve this problem,we designed a graph structure adaptive learning layer based on neural networks,which can automatically learn the graph structure from time series data.It not only better captures the relationships between nodes but also enhances the model’s performance by augmenting key data.Experiments have demonstrated that our method improves the baseline model with the highest F1 value by 10.4%,36.1%,and 5.6%,respectively,on three real datasets with a 10%data labeling rate.

Model Change Active Learning in Graph-Based Semi-supervised Learning

Active learning in semi-supervised classification involves introducing additional labels for unlabelled data to improve the accuracy of the underlying classifier.A challenge is to identify which points to label to best improve performance while limiting the number of new labels."Model Change"active learning quantifies the resulting change incurred in the classifier by introducing the additional label(s).We pair this idea with graph-based semi-supervised learning(SSL)methods,that use the spectrum of the graph Laplacian matrix,which can be truncated to avoid prohibitively large computational and storage costs.We consider a family of convex loss functions for which the acquisition function can be efficiently approximated using the Laplace approximation of the posterior distribution.We show a variety of multiclass examples that illustrate improved performance over prior state-of-art.

Lateral interaction by Laplacian‐based graph smoothing for deep neural networks

Lateral interaction in the biological brain is a key mechanism that underlies higher cognitive functions.Linear self‐organising map(SOM)introduces lateral interaction in a general form in which signals of any modality can be used.Some approaches directly incorporate SOM learning rules into neural networks,but incur complex operations and poor extendibility.The efficient way to implement lateral interaction in deep neural networks is not well established.The use of Laplacian Matrix‐based Smoothing(LS)regularisation is proposed for implementing lateral interaction in a concise form.The authors’derivation and experiments show that lateral interaction implemented by SOM model is a special case of LS‐regulated k‐means,and they both show the topology‐preserving capability.The authors also verify that LS‐regularisation can be used in conjunction with the end‐to‐end training paradigm in deep auto‐encoders.Additionally,the benefits of LS‐regularisation in relaxing the requirement of parameter initialisation in various models and improving the classification performance of prototype classifiers are evaluated.Furthermore,the topologically ordered structure introduced by LS‐regularisation in feature extractor can improve the generalisation performance on classification tasks.Overall,LS‐regularisation is an effective and efficient way to implement lateral interaction and can be easily extended to different models.

Unfolding the structure-property relationships of Li_(2)S anchoring on two-dimensional materials with high-throughput calculations and machine learning

Lithium-sulfur(Li-S)batteries are notable for their high theoretical energy density,but the‘shuttle effect’and the limited conversion kinetics of Li-S species can downgrade their actual performance.An essential strategy is to design anchoring materials(AMs)to appropriately adsorb Li-S species.Herein,we propose a new three-procedure protocol,named InfoAd(Informative Adsorption)to evaluate the anchoring of Li_(2)S on two-dimensional(2D)materials and disclose the underlying importance of material features by combining high-throughput calculation workflow and machine learning(ML).In this paradigm,we calculate the anchoring of Li_(2)S on 12552D A_(x)B_(y)(B in the VIA/VIIA group)materials and pick out 44(un)reported nontoxic 2D binary A_(x)B_(y)AMs,in which the importance of the geometric features on the anchoring effect is revealed by ML for the first time.We develop a new Infograph model for crystals to accurately predict whether a material has a moderate binding with Li_(2)S and extend it to all 2D materials.Our InfoAd protocol elucidates the underlying structure-property relationship of Li_(2)S adsorption on 2D materials and provides a general research framework of adsorption-related materials for catalysis and energy/substance storage.

Improving link prediction models through a performance enhancement scheme:a study on semi-supervised learning and model soup

As an important method for knowledge graph(KG)complementation,link prediction has become a hot research topic in recent years.In this paper,a performance enhancement scheme for link prediction models based on the idea of semi-supervised learning and model soup is proposed,which effectively improves the model performance on several mainstream link prediction models with small changes to their architecture.This novel scheme consists of two main parts,one is predicting potential fact triples in the graph with semi-supervised learning strategies,the other is creatively combining semi-supervised learning and model soup to further improve the final model performance without adding significant computational overhead.Experiments validate the effectiveness of the scheme for a variety of link prediction models,especially on the dataset with dense relationships.In terms of CompGCN,the model with the best overall performance among the tested models improves its Hits@1 metric by 14.7%on the FB15K-237 dataset and 7.8%on the WN18RR dataset after using the enhancement scheme.Meanwhile,it is observed that the semi-supervised learning strategy in the augmentation scheme has a significant improvement for multi-class link prediction models,and the performance improvement brought by the introduction of the model soup is related to the specific tested models,as the performances of some models are improved while others remain largely unaffected.

Instance selection method for improving graph-based semi-supervised learning

Graph-based semi-supervised learning is an important semi-supervised learning paradigm. Although graphbased semi-supervised learning methods have been shown to be helpful in various situations, they may adversely affect performance when using unlabeled data. In this paper, we propose a new graph-based semi-supervised learning method based on instance selection in order to reduce the chances of performance degeneration. Our basic idea is that given a set of unlabeled instances, it is not the best approach to exploit all the unlabeled instances; instead, we should exploit the unlabeled instances that are highly likely to help improve the performance, while not taking into account the ones with high risk. We develop both transductive and inductive variants of our method. Experiments on a broad range of data sets show that the chances of performance degeneration of our proposed method are much smaller than those of many state-of-the-art graph-based semi-supervised learning methods.

Multi-Domain Malicious Behavior Knowledge Base Framework for Multi-Type DDoS Behavior Detection

Due to the many types of distributed denial-of-service attacks(DDoS)attacks and the large amount of data generated,it becomes a chal-lenge to manage and apply the malicious behavior knowledge generated by DDoS attacks.We propose a malicious behavior knowledge base framework for DDoS attacks,which completes the construction and application of a multi-domain malicious behavior knowledge base.First,we collected mali-cious behavior traffic generated by five mainstream DDoS attacks.At the same time,we completed the knowledge collection mechanism through data pre-processing and dataset design.Then,we designed a malicious behavior category graph and malicious behavior structure graph for the characteristic information and spatial structure of DDoS attacks and completed the knowl-edge learning mechanism using a graph neural network model.To protect the data privacy of multiple multi-domain malicious behavior knowledge bases,we implement the knowledge-sharing mechanism based on federated learning.Finally,we store the constructed knowledge graphs,graph neural network model,and Federated model into the malicious behavior knowledge base to complete the knowledge management mechanism.The experimental results show that our proposed system architecture can effectively construct and apply the malicious behavior knowledge base,and the detection capability of multiple DDoS attacks occurring in the network reaches above 0.95,while there exists a certain anti-interference capability for data poisoning cases.

多智能体深度强化学习研究进展

深度强化学习(Deep Reinforcement Learning,DRL)在近年受到广泛的关注,并在各种领域取得显著的成功.由于现实环境通常包括多个与环境交互的智能体,多智能体深度强化学习(Multi-Agent Deep Reinforcement Learning,MADRL)获得蓬勃的发展,在各种复杂的序列决策任务上取得优异的表现.本文对多智能体深度强化学习的工作进展进行综述,主要内容分为三个部分.首先,我们回顾了几种常见的多智能体强化学习问题表示及其对应的合作、竞争和混合任务.其次,我们对目前的MADRL方法进行了全新的多维度的分类,并对不同类别的方法展开进一步介绍.其中,我们重点综述值函数分解方法,基于通信的MADRL方法以及基于图神经网络的MADRL方法.最后,我们研究了MADRL方法在现实场景中的主要应用.希望本文能够为即将进入这一快速发展领域的新研究人员和希望获得全方位了解并根据最新进展确定新方向的现有领域专家提供帮助.

联合句法与位置信息的方面情感三元组抽取

为提高方面级情感三元组抽取任务的准确率,提出一种联合依存句法关系和位置偏移信息的抽取模型。在模型上下文编码中添加句法关系,结合图卷积网络捕获结构和结点属性信息,增强三元组要素之间的交互能力;在多任务学习部分加入相对位置偏移信息,充分挖掘方面-观点词对的关系,提高三元组要素抽取的精度。在4个基准英文数据集上的实验结果表明,该方法效果显著且优于其它基线模型。

大语言模型与知识图谱协同下高校AI项目导师构建研究——以“人工智能创新项目设计”为例

针对项目式学习面临的学习内容增加、知识体系重构以及学习方式变化等难题,提出了一种融合大语言模型(LLM)与知识图谱技术的高校AI项目导师系统构建方案。其中,大语言模型实现自然语言的理解和生成,而知识图谱负责专业知识库和增强大语言模型输出的可解释性和可靠性。高校AI项目导师能提供准确、个性化且全面的项目学习指导,使学习者有效应对各种复杂的项目式学习情境。“人工智能创新项目设计”课程的实证研究表明,采用高校AI项目导师系统显著提升了学生的课程参与度,促进了学生的工程创新素养、工程人文素养及AI素养三大核心素养的发展,并有效提高了学生对课程的满意度。这表明将大语言模型技术与知识图谱技术相结合而开发的在线教育工具,在提升教学质量、激发创新能力等方面展现出较高的发展潜力。

基于依赖类型剪枝的双特征自适应融合网络用于方面级情感分析

现有的模型将基于依赖树的图神经网络用于方面级情感分析,一定程度上提升了模型的分类性能。然而,由于依赖解析技术的限制,语法解析结果的不精确导致依赖树存在大量噪声,使得模型的性能提升有限。此外,一些句子本身并不符合标准的句法结构。以往的研究以同样的置信度利用句法信息和语义信息,没有充分考虑它们对于确定方面词极性的贡献的不同,导致模型在相应的数据集上性能较差。为了克服这些困难,文中提出了一种基于依赖类型剪枝的双特征自适应融合网络。具体来说,该模型使用一种新型的混合方法,命名为依赖关系类型剪枝和邻接矩阵平滑,来缓解句法解析产生的噪声。此外,该模型通过双特征自适应融合模块充分考虑句子的句法信息的可用程度,以一种更灵活的方式将句法特征和语义特征结合起来用于方面级情感分析。在5个公开可用的数据集上进行广泛的实验,结果证明了该方法明显优于基线模型。

NGAT:attention in breadth and depth exploration for semi-supervised graph representation learning

Recently,graph neural networks(GNNs)have achieved remarkable performance in representation learning on graph-structured data.However,as the number of network layers increases,GNNs based on the neighborhood aggregation strategy deteriorate due to the problem of oversmoothing,which is the major bottleneck for applying GNNs to real-world graphs.Many efforts have been made to improve the process of feature information aggregation from directly connected nodes,i.e.,breadth exploration.However,these models perform the best only in the case of three or fewer layers,and the performance drops rapidly for deep layers.To alleviate oversmoothing,we propose a nested graph attention network(NGAT),which can work in a semi-supervised manner.In addition to breadth exploration,a k-layer NGAT uses a layer-wise aggregation strategy guided by the attention mechanism to selectively leverage feature information from the k;-order neighborhood,i.e.,depth exploration.Even with a 10-layer or deeper architecture,NGAT can balance the need for preserving the locality(including root node features and the local structure)and aggregating the information from a large neighborhood.In a number of experiments on standard node classification tasks,NGAT outperforms other novel models and achieves state-of-the-art performance.

基于知识图谱的学习系统设计对在线学习效果的影响研究

从建构主义和能力本位理论出发,提出了一种基于知识图谱的在线学习系统设计方法,即打破传统的知识结构,以提升能力为目标,构建知识、技能等多维度的能力框架;搭建了以知识图谱为底层逻辑,链接数字学习资源的学习系统;开展了教学实践和实证研究.首先,使用调查问卷对学习系统进行了验证;其次,以“阅读英文学术论文”能力为学习任务,随机分配实验组和对照组;最后,评估两组对于知识、技能的理解、记忆水平以及综合运用能力.研究结果显示,学习系统的有效性和易用性实验组总成绩、知识得分、技能得分和能力得分均高于对照组成绩,其中总成绩和能力得分具有显著性差异,表明该系统对于在线学习效果有一定的促进作用.

基于超图卷积网络和目标多意图感知的会话推荐算法

当前先进的会话推荐算法主要通过图神经网络从全局和目标会话中挖掘项目的成对转换关系,并将目标会话压缩成固定的向量表示,忽略了项目间复杂的高阶信息和目标项目对用户偏好多样性的影响。为此提出了基于超图卷积网络和目标多意图感知的会话推荐算法HCN-TMP。通过学习会话表示来表达用户偏好,首先依据目标会话构建会话图,依据全局会话构建超图,通过意图解纠缠技术将原有反映用户耦合意图的项目嵌入表示转换为项目多因素嵌入表示,再经图注意力网络和超图卷积网络分别学习目标会话节点的会话级和全局级项目表示,并使用距离相关性损失函数增强多因素嵌入块间的独立性;然后嵌入目标会话中节点位置信息,加权每个节点的注意力权重,得到全局级和会话级会话表示;利用对比学习最大化两者互信息,经目标多意图感知,针对不同的目标项目自适应地学习目标会话中多意图的用户偏好,得到目标感知级会话表示,最后线性融合三个级别的会话表示得到最终的会话表示。在Tmall和Nowplaying两个公开数据集上进行大量实验,实验结果验证了HCN-TMP算法的有效性。

基于图辅助学习的会话推荐

针对现有的自监督对比任务未能充分利用原始数据中的丰富语义以及缺乏通用性的问题,提出一种基于图辅助学习的会话推荐(SR-GAL)模型。首先,在图神经网络(GNN)的基础上引入具有表示一致性(RC)的编码通道,从原始数据中挖掘更有价值的自监督信号;其次,为了充分利用这些自监督信号,设计了与目标任务关系紧密的预测性辅助任务和约束性辅助任务;最后,开发了一个简单且与GNN模型无关的辅助学习框架,将两个辅助任务与推荐任务统一起来,从而提高GNN模型的推荐性能。与次优的对比模型CGSNet(Contrastive Graph Self-attention Network)相比,在Diginetica数据集上,所提模型的精确率P@20和平均倒数排名MRR@20提升了0.58%和1.61%;在Tmall数据集上,所提的模型的P@20和MRR@20分别提升了12.65%和8.41%,验证了该模型的有效性。在多个真实数据集上的实验结果表明,SR-GAL模型优于较先进的模型,并且具有良好的可扩展性和通用性。

自监督混合图神经网络的会话推荐模型

基于会话的推荐旨在利用匿名会话预测用户行为。现有基于图神经网络(GNN)的会话推荐算法大多仅针对当前会话提取用户偏好,却忽略了来自其他会话的高阶多元关系从而影响推荐精度。此外,由于会话推荐所采用的短时交互序列包含的信息非常有限,使其更容易受到数据稀疏性的影响。针对上述问题,提出了自监督混合图神经网络会话推荐模型(SHGN)。该模型首先通过将原始数据构建为三个视图来描述会话与物品关系,然后通过多头图注意力网络捕获会话内部物品的低阶转换信息,提出了残差图卷积网络捕获物品和会话的高阶转换信息;最后融合自监督学习(SSL)作为辅助任务,通过最大化不同通道学习到的会话嵌入的互信息,对原始数据进行数据增强从而提升推荐性能。为了验证该方法的有效性,在Tmall、Diginetica、Nowplaying、Yoochoose四个基准数据集上与SR-GNN、GCE-GNN、DHCN等主流基线模型进行了对比实验,实验结果在P@20、MRR@20等性能指标上均取得了一定提升。

基于句信息增强词信息的方面级情感分类

方面级情感分类属于细粒度的情感分类,旨在判断句子中指定方面词的情感极性。近年来,句法知识在方面级情感分类任务中得到了广泛应用。目前主流的模型利用句法依存树和图卷积神经网络进行情感极性的分类。然而,此类模型着眼于利用聚合后的方面词信息来判断情感极性,很少关注句子的全局信息对情感极性的影响,从而导致情感极性分类结果出现偏差。为了解决这一问题,提出了一种基于句信息增强词信息的方面级情感分类模型,该模型通过对比学习对句向量进行表示学习,以减小句向量对比损失为目标调整词向量的特征表示,最后通过图卷积神经网络聚合意见词信息得出情感分类结果。在SemEval2014数据集和Twitter数据集上进行实验,结果表明,所提模型可以提高分类的准确性,综合验证了该方法的有效性。

联合对比学习的图神经网络会话推荐

基于会话的推荐(SBR)是一项具有挑战性的任务,其目的是根据匿名行为序列推荐项目。本文提出了一种新的方法,称为联合对比学习的图神经网络会话推荐(CLGNN),在图注意力机制的基础上,用对比学习辅助训练,以获得更好的推荐结果。具体来说,CLGNN首先在会话图上采用注意力机制学习项目嵌入,然后聚合会话内的项目生成会话嵌入,最后使用会话嵌入和候选项目嵌入计算分数生成推荐,同时使用对比学习优化项目嵌入空间。以几种常见的评价指标为依据,在真实的两个数据集上进行实验,结果表明本文模型推荐性能良好。

Leveraging graph convolutional networks for semi-supervised fault diagnosis of HVAC systems in data-scarce contexts

The continuous accumulation of operational data has provided an ideal platform to devise and implement customized data analytics for smart HVAC fault detection and diagnosis.In practice,the potentials of advanced supervised learning algorithms have not been fully realized due to the lack of sufficient labeled data.To tackle such data challenges,this study proposes a graph neural network-based approach to effectively utilizing both labeled and unlabeled operational data for optimum decision-makings.More specifically,a graph generation method is proposed to transform tabular building operational data into association graphs,based on which graph convolutions are performed to derive useful insights for fault classifications.Data experiments have been designed to evaluate the values of the methods proposed.Three datasets on HVAC air-side operations have been used to ensure the generalizability of results obtained.Different data scenarios,which vary in training data amounts and imbalance ratios,have been created to comprehensively quantify behavioral patterns of representative graph convolution networks and their architectures.The research results indicate that graph neural networks can effectively leverage associations among labeled and unlabeled data samples to achieve an increase of 2.86%–7.30%in fault classification accuracies,providing a novel and promising solution for smart building management.

基于对齐性和均匀性约束的图神经网络会话推荐方法

会话推荐(session-based recommendation,SBR)旨在匿名状态下通过用户的短期历史行为序列来预测下一个待点击的项目。为解决现有基于图神经网络(graph neural networks,GNNs)的会话推荐方法忽略会话中不同位置相同项目之间差异的问题,在图卷积获得项目表示后,进一步考虑该项目与相邻项目之间的相关性,生成邻域相关的项目表示。此外,鉴于对齐性和均匀性在对比学习中的起到的重要作用,还提出了一种适用于会话推荐的对齐性和均匀性损失方法,以约束生成的会话表示和项目表示。在3个公开数据集上的实验表明,文中提出的模型TAU-GNN的推荐性能优于对比的主流会话推荐模型。