GNNSched:面向GPU的图神经网络推理任务调度框架

由于频繁的显存访问,图神经网络GNN在GPU上运行时往往资源利用率较低。现有的推理框架由于没有考虑GNN输入的不规则性,直接适用到GNN进行推理任务共置时可能会超出显存容量导致任务失败。对于GNN推理任务,需要根据其输入特点预先分析并发任务的显存占用情况,以确保并发任务在GPU上的成功共置。此外,多租户场景提交的推理任务亟需灵活的调度策略,以满足并发推理任务的服务质量要求。为了解决上述问题,提出了GNNSched,其在GPU上高效管理GNN推理任务的共置运行。具体来说,GNNSched将并发推理任务组织为队列,并在算子粒度上根据成本函数估算每个任务的显存占用情况。GNNSched实现了多种调度策略来生成任务组,这些任务组被迭代地提交到GPU并发执行。实验结果表明,GNNSched能够满足并发GNN推理任务的服务质量并降低推理任务的响应时延。

基于强化联邦GNN的个性化公共安全突发事件检测

近年来,将公共安全数据转换为图的形式,通过图神经网络(GNN)构造节点表示应用于下游任务的方法,充分利用了公共安全数据的实体与关联信息,取得了较好的效果.为了提高模型的有效性,需要大量的高质量数据,但是高质量的数据通常归属于政府、公司和组织,很难通过数据集中的方式使模型学习到有效的事件检测模型.由于各数据拥有方的关注主题与收集时间不同,数据之间存在Non-IID的问题.传统的假设一个全局模型可以适合所有客户端的方法难以解决此类问题.提出了基于强化联邦图神经网络的个性化公共安全突发事件检测方法PPSED,各客户端采用多方协作的方式训练个性化的模型来解决本地的突发事件检测任务.设计了联邦公共安全突发事件检测模型的本地训练与梯度量化模块,采用基于图采样的minibatch机制的GraphSage构造公共安全突发事件检测本地模型,以减小数据Non-IID的影响,采用梯度量化方法减小梯度通信的消耗.设计了基于随机图嵌入的客户端状态感知模块,在保护隐私的同时,更好地保留客户端模型有价值的梯度信息.设计了强化联邦图神经网络的个性化梯度聚合与量化策略,采用DDPG拟合个性化联邦学习梯度聚合加权策略,并根据权重决定是否对梯度进行量化,对模型的性能与通信压力进行平衡.通过在微博平台收集的公共安全数据集和3个公开的图数据集进行了大量的实验,实验结果表明了所提方法的有效性.

Hybridized Intelligent Neural Network Optimization Model for Forecasting Prices of Rubber in Malaysia

Rubber producers,consumers,traders,and those who are involved in the rubber industry face major risks of rubber price fluctuations.As a result,decision-makers are required to make an accurate estimation of the price of rubber.This paper aims to propose hybrid intelligent models,which can be utilized to forecast the price of rubber in Malaysia by employing monthly Malaysia’s rubber pricing data,spanning from January 2016 to March 2021.The projected hybrid model consists of different algorithms with the symbolic Radial Basis Functions Neural Network k-Satisfiability Logic Mining(RBFNN-kSAT).These algorithms,including Grey Wolf Optimization Algorithm,Artificial Bee Colony Algorithm,and Particle Swarm Optimization Algorithm were utilized in the forecasting data analysis.Several factors,which affect the monthly price of rubber,such as rubber production,total exports of rubber,total imports of rubber,stocks of rubber,currency exchange rate,and crude oil prices were also considered in the analysis.To evaluate the results of the introduced model,a comparison has been conducted for each model to identify the most optimum model for forecasting the price of rubber.The findings showed that GWO with RBFNN-kSAT represents the most accurate and efficient model compared with ABC with RBFNNkSAT and PSO with RBFNN-kSAT in forecasting the price of rubber.The GWO with RBFNN-kSAT obtained the greatest average accuracy(92%),with a better correlation coefficient R=0.983871 than ABC with RBFNN-kSAT and PSO with RBFNN-kSAT.Furthermore,the empirical results of this study provided several directions for policymakers to make the right decision in terms of devising proper measures in the industry to address frequent price changes so that the Malaysian rubber industry maintains dominance in the international markets.

ELMAN Neural Network with Modified Grey Wolf Optimizer for Enhanced Wind Speed Forecasting

The scope of this paper is to forecast wind speed. Wind speed, temperature, wind direction, relative humidity, precipitation of water content and air pressure are the main factors make the wind speed forecasting as a complex problem and neural network performance is mainly influenced by proper hidden layer neuron units. This paper proposes new criteria for appropriate hidden layer neuron unit’s determination and attempts a novel hybrid method in order to achieve enhanced wind speed forecasting. This paper proposes the following two main innovative contributions 1) both either over fitting or under fitting issues are avoided by means of the proposed new criteria based hidden layer neuron unit’s estimation. 2) ELMAN neural network is optimized through Modified Grey Wolf Optimizer (MGWO). The proposed hybrid method (ELMAN-MGWO) performance, effectiveness is confirmed by means of the comparison between Grey Wolf Optimizer (GWO), Adaptive Gbest-guided Gravitational Search Algorithm (GGSA), Artificial Bee Colony (ABC), Ant Colony Optimization (ACO), Cuckoo Search (CS), Particle Swarm Optimization (PSO), Evolution Strategy (ES), Genetic Algorithm (GA) algorithms, meanwhile proposed new criteria effectiveness and precise are verified comparison with other existing selection criteria. Three real-time wind data sets are utilized in order to analysis the performance of the proposed approach. Simulation results demonstrate that the proposed hybrid method (ELMAN-MGWO) achieve the mean square error AVG ± STD of 4.1379e-11 ± 1.0567e-15, 6.3073e-11 ± 3.5708e-15 and 7.5840e-11 ± 1.1613e-14 respectively for evaluation on three real-time data sets. Hence, the proposed hybrid method is superior, precise, enhance wind speed forecasting than that of other existing methods and robust.

A study of using grey system theory and artificial neural network on the climbing ability of <i>Buergeria robusta</i>frog

Ecological engineering is an emerging study of integrating both ecology and engineering, concerned with the design, monitoring, and construction of ecosystems. In recent years, the threat to amphibian animals is becoming more and more serious. In particular, the loss of habitats caused by changes to the way land is used by human beings has hit amphibians particularly hard. Amphibians are known to be particularly vulnerable to human activities because they rely on both terrestrial and aquatic habitats for survival. With the increasing development of many areas in recent years, concrete structures are often installed along water bodies in order to increase the safety of local residents. The construction of concrete banks along rivers associated with human development has become a serious problem in Taiwan. Most ecosystems used by amphibians are lakes and stream banks, yet no related design solutions to accommodate the needs of amphibians. The need to develop the relevant design specification considering protecting the amphibian is imperative. Buergeria robusta, an endemic species in Taiwan, is tree frog widely distributed in lowland montane regions. Their breeding season is from April to September. They like to rest on trees or hide at caves during the daytime and move to the stream nearby in dusk for breeding. Males usually emit weak mating call while standing on stones. Sticky eggs are attached to undersides of rocks and stones. Tadpoles are found in slow flowing water of streams [1]. The goal of this study is to improve the understanding of the relationship between the climbing ability and the physical characteristics of amphibians. In this study, we use Artificial Neural Network to simulate the climbing ability of Buergeria robusta. Besides, Grey System Theory is also adopted to improve the performance of Artificial Neural Network. Artificial Neural Network (ANN) is a computing system that uses a large number of artificial neurons imitating natural neural ability to deal with an information network by computing system. The numerical results have show good agreement with the experimental results. The results can serve as a reference for technicians involved in future ecological engineering designs of banks throughout the world.

结合注意力CNN与GNN的信息融合推荐方法

稀疏性问题一直是推荐系统面临的主要挑战,而信息融合推荐可以利用用户的评论、评分以及信任等信息发掘用户的偏好来缓解这一问题,从而为目标用户生成相应的推荐.用户、项目信息的充分学习是构建一个成功推荐系统的关键.但不同用户对不同项目有不同的偏好,且用户的兴趣偏好及社交圈是动态变化的.提出一种结合深度学习与信息融合的推荐方法来解决稀疏性等问题.特别地,构建了一种新的深度学习模型——结合注意力卷积神经网络(attention CNN)与图神经网络(GNN)的信息融合推荐模型ACGIF.首先,在CNN中加入注意力机制来处理评论信息,从评论信息中学习用户和项目的个性化表示.根据评论编码学习评论表示,通过用户/项目编码学习评论中用户/项目表示.加入个性化注意力机制来筛选不同重要性级别的评论.然后,利用GNN来处理评分和信任信息.对于每个用户来说,扩散过程从最初的嵌入开始,融合相关特性和捕获潜在行为偏好的自由用户潜在向量.设计了一个分层的影响传播结构,以模拟用户的潜在嵌入如何随着社交扩散过程的继续而演变.最后,对前两部分得到的用户对项目的偏好向量进行加权融合,获得最终的用户对于项目的偏好向量.在4组公开数据集上,以推荐结果的MAE和RMSE作为评估指标进行了实验验证.结果表明,与现有的7个典型推荐模型相比,所提模型的推荐效果和运行时间均占优.

Design of Neural Network Based Wind Speed Prediction Model Using GWO

The prediction of wind speed is imperative nowadays due to the increased and effective generation of wind power.Wind power is the clean,free and conservative renewable energy.It is necessary to predict the wind speed,to implement wind power generation.This paper proposes a new model,named WT-GWO-BPNN,by integrating Wavelet Transform(WT),Back Propagation Neural Network(BPNN)and GreyWolf Optimization(GWO).The wavelet transform is adopted to decompose the original time series data(wind speed)into approximation and detailed band.GWO-BPNN is applied to predict the wind speed.GWO is used to optimize the parameters of back propagation neural network and to improve the convergence state.This work uses wind power data of six months with 25,086 data points to test and verify the performance of the proposed model.The proposed work,WT-GWO-BPNN,predicts the wind speed using a three-step procedure and provides better results.Mean Absolute Error(MAE),Mean Squared Error(MSE),Mean absolute percentage error(MAPE)and Root mean squared error(RMSE)are calculated to validate the performance of the proposed model.Experimental results demonstrate that the proposed model has better performance when compared to other methods in the literature.

Analysis and Prediction of Regional Electricity Consumption Based on BP Neural Network

Electricity consumption forecasting is one of the most important tasks for power system workers,and plays an important role in regional power systems.Due to the difference in the trend of power load and the past in the new normal,the influencing factors are more diversified,which makes it more difficult to predict the current electricity consumption.In this paper,the grey system theory and BP neural network are combined to predict the annual electricity consumption in Jiangsu.According to the historical data of annual electricity consumption and the six factors affecting electricity consumption,the gray correlation analysis method is used to screen the important factors,and three factors with large correlation degree are selected as the input parameters of BP neural network.The power forecasting model uses nearly 18 years of data to train and validate the model.The results show that the gray correlation analysis and BP neural network method have higher accuracy in power consumption prediction,and the calculation is more convenient than traditional methods.

Grey Wolf Optimizer Based Deep Learning for Pancreatic Nodule Detection

At an early point,the diagnosis of pancreatic cancer is mediocre,since the radiologist is skill deficient.Serious threats have been posed due to the above reasons,hence became mandatory for the need of skilled technicians.However,it also became a time-consuming process.Hence the need for automated diagnosis became mandatory.In order to identify the tumor accurately,this research pro-poses a novel Convolution Neural Network(CNN)based superior image classi-fication technique.The proposed deep learning classification strategy has a precision of 97.7%,allowing for more effective usage of the automatically exe-cuted feature extraction technique to diagnose cancer cells.Comparative analysis with CNN-Grey Wolf Optimization(GWO)is carried based on varied testing and training outcomes.The suggested study is carried out at a rate of 90%–10%,80%–20%,and 70%–30%,indicating the robustness of the proposed research work.Outcomes show that the suggested method is effective.GWO-CNN is reli-able and accurate relative to other detection methods available in the literatures.

Recognizing Ancient South Indian Language Using Opposition Based Grey Wolf Optimization

Recognizing signs and fonts of prehistoric language is a fairly difficult job that requires special tools.This stipulation make the dispensation period over-riding,difficult and tiresome to calculate.This paper present a technique for recognizing ancient south Indian languages by applying Artificial Neural Network(ANN)associated with Opposition based Grey Wolf Optimization Algorithm(OGWA).It identifies the prehistoric language,signs and fonts.It is an apparent from the ANN system that arbitrarily produced weights or neurons linking various layers play a significant role in its performance.For adaptively determining these weights,this paper applies various optimization algorithms such as Opposition based Grey Wolf Optimization,Particle Swarm Optimization and Grey Wolf Opti-mization to the ANN system.Performance results are illustrated that the proposed ANN-OGWO technique achieves superior accuracy over the other techniques.In test case 1,the accuracy value of OGWO is 94.89%and in test case 2,the accu-racy value of OGWO is 92.34%,on average,the accuracy of OGWO achieves 5.8%greater accuracy than ANN-GWO,10.1%greater accuracy than ANN-PSO and 22.1%greater accuracy over conventional ANN technique.

基于邻域采样的多任务图推荐算法

近年来,图神经网络(GNN)成为解决协同过滤的主流方法之一。它通过构建用户-物品图,模拟用户与物品的交互关系,并用GNN学习它们的特征表示。尽管现有在模型结构上的研究已取得了较大进展,但如何在图结构上更有效地进行负采样仍未有效解决。为此,提出一种基于邻域采样的多任务图推荐算法。该算法提出了一种基于GNN的邻域采样策略,该策略以每个用户为中心构建子图,将次高阶物品作为用户邻域采样的负样本,可以更有效地挖掘强负样本并提高采样质量。通过GNN对图结点进行信息聚合与特征提取,得到结点的最终嵌入表示。设计一种余弦边际损失来过滤部分冗余负样本,以有效减少采样过程中的噪声数据。同时,该算法引入了多任务策略对模型进行联合优化,以增强模型的泛化能力。在3个公开数据集上进行的大量实验表明,该算法在大多数情况下明显优于其他主流算法。

基于图神经网络的人工自然语言语义挖掘仿真

语义挖掘工具可从批量非结构化人工自然语言文本数据中准确提取有用信息,但是由于网络环境文本具备半结构化、多尺度、海量、复杂关联等属性,导致文本数据通常维度较高,且仅有小部分节点存在明确标签,因此语义挖掘难度较大。提出基于图神经网络的人工自然语言语义挖掘方法。结合多头注意力机制和半监督图卷积神经网络对人工自然语言文本降维处理。联合改进的模糊C均值聚类算法和免疫单亲遗传算法,构建人工自然语言语义挖掘算法。实验结果表明,研究方法的聚类纯度、准确率和召回率均高于95%,说明上述方法的应用性能较优。

基于多模态融合的图神经网络推荐算法

已有的图神经网络(GNN)推荐算法大多利用用户-项目交互图的节点编号信息进行训练,学习用户-项目节点的高阶联系去丰富节点表示,但忽略了用户对不同模态信息的偏好,没有利用项目的图片、文本等模态信息,或对于不同模态特征的融合简单相加,不能区分用户对不同模态信息的偏好。针对上述问题,提出多模态融合的GNN推荐模型。首先针对单个模态,结合用户-项目交互二部图构建单模态图网络,在单模态图中学习用户对此模态信息的偏好;然后利用GAT聚合邻居信息,丰富本节点表示,同时根据门控循环单元决定是否聚合邻居信息,达到去噪效果;最后将各个模态图学习到的用户、项目表示通过注意力机制融合得到最终表示并送入预测模块。在MovieLens-20M、H&M两个数据集上的实验结果表明:多模态信息、注意力融合机制能有效提升推荐的准确度,算法模型在Precision@K、Recall@K和NDCG@K 3个指标上相较于基线最优算法均有显著提升;当评估指标K值选取10时,Precision@10、Recall@10和NDCG@10在两个数据集上分别提升了4.67%、2.42%、2.03%和2.49%、5.24%、2.05%。

KENN:线性结构熵的图核神经网络

图神经网络(graph neural network,GNN)是一种利用深度学习直接对图结构数据进行表征的框架,近年来受到人们越来越多的关注.然而传统的基于消息传递聚合的图神经网络(messaging passing GNN,MP-GNN)忽略了不同节点的平滑速度,无差别地聚合了邻居信息,易造成过平滑现象.为此,研究并提出一种线性结构熵的图核神经网络分类方法,即KENN.它首先利用图核方法对节点子图进行结构编码,判断子图之间的同构性,进而利用同构系数来定义不同邻居间的平滑系数.其次基于低复杂度的线性结构熵提取图的结构信息,加深和丰富图数据的结构表达能力.通过将线性结构熵、图核和图神经网络三者进行深度融合提出了图核神经网络分类方法.它不仅可以解决生物分子数据节点特征的稀疏问题,也可以解决社交网络数据以节点度作为特征所产生的信息冗余问题,同时还使得图神经网络能够自适应调整对图结构特征的表征能力,使其超越MP-GNN的上界(WL测试).最后,在7个公开的图分类数据集上实验验证了所提出模型的性能优于其他的基准模型.

邻域信息分层感知的知识图谱补全方法

知识图谱补全(KGC)旨在利用知识图谱的现有知识推断三元组的缺失值。近期的一些研究表明,将图卷积网络(GCN)应用于KGC任务有助于改善模型的推理性能。针对目前大多数GCN模型存在的同等对待邻域信息、忽略了邻接实体对中心实体的不同贡献度、采用简单的线性变换更新关系嵌入等问题,提出了一个邻域信息分层感知的图神经网络模型NAHAT,在关系更新中引入实体特征信息,通过聚合实体和关系表征来丰富异质关系语义,提高模型的表达能力。同时,将自我对立的负样本训练应用到损失计算中,实现模型的高效训练。实验结果表明,与图卷积网络模型COMPGCN相比,所提出的模型在FB15K-237数据集上Hits@1、Hits@10指标分别提高了3%、2.6%;在WN18RR数据集上分别提高了0.9%、2.2%。验证了所提出的模型的有效性。

基于自监督的多视角图协同过滤推荐方法

现有的图协同过滤算法在现实场景中存在数据稀疏问题,同时在相邻信息聚合的过程中使得特征学习更容易受到交互噪声的影响。为了解决上述问题,提出一个基于自监督的多视角图协同过滤(SMGCF)推荐方法,通过图神经网络学习用户和项目节点的嵌入表示。在学习节点嵌入表示的过程中,考虑到单个节点间的交互关系以及聚类节点间的聚类关系对推荐结果的影响,引入自监督学习来辅助图协同过滤算法进行多视角关系的挖掘。针对节点交互级关系视角,通过数据增强得到多个用户-项目交互二分图,并且提出一种节点交互级关系的对比学习方法;针对节点聚类级关系视角,提出一种节点聚类级关系的对比学习方法。通过多视角融合策略将2种类型的对比学习方法进行融合,从而提升节点嵌入效果。在4个公开的数据集上进行实验,实验结果证明了SMGCF的可行性和有效性。相比最优基准方法NCL,SMGCF在Recall@10和NDCG@10指标上最高可提升2.1%和4.3%。

基于角度的图神经网络高维数据异常检测方法

在高维数据空间中,数据大都处于高维空间边缘且分布十分稀疏,由此引起的“维度灾难”问题导致现有异常检测方法无法保证异常检测精度。为解决该问题,提出一种基于角度的图神经网络高维数据异常检测方法A-GNN。首先通过数据空间的均匀采样和初始训练数据的扰动来扩充用于训练的数据;然后利用k近邻关系构造训练数据的k近邻关系图,并以k近邻元素距离加权角度的方差作为近邻关系图节点的初始异常因子;最后通过训练图神经网络模型,实现节点间的信息交互,使得相邻节点能够互相学习,从而进行有效的异常评估。在6个自然数据集上将A-GNN方法与9种典型异常检测方法进行实验对比,结果表明:A-GNN在5个数据集中取得了最高的AUC值,其能够大幅提升各种维度数据的异常检测精度,在一些“真高维数据”上异常检测的AUC值提升达40%以上;在不同k值下与3种基于k近邻的异常检测方法相比,A-GNN利用图神经网络节点间的信息交互能有效避免k值对检测结果的影响,方法具有更强的鲁棒性。

面向节点分类任务的节点级自适应图卷积神经网络

图神经网络通过对图中节点的递归采样与聚合以学习节点嵌入,而现有方法中节点采样与聚合的模式较固定,对局部模式的多样性捕获存在不足,从而降低模型性能.因此,文中提出节点级自适应图卷积神经网络(Node-Level Adaptive Graph Convolutional Neural Network,NA-GCN).设计基于节点重要性的采样策略,自适应地确定各节点的邻域规模.同时,提出基于自注意力机制的聚合策略,自适应地融合给定邻域内的节点信息.在多个基准图数据集上的实验表明,NA-GCN在节点分类任务上具有较优性能.

基于输入特征稀疏化的图神经网络训练加速

图神经网络(graph neural network,GNN)是处理图数据的重要方法.由于计算复杂、图数据容量大,在大规模图上训练图神经网络依赖于CPU-GPU协作和图采样训练方法,其中图结构和特征数据存储在CPU内存中,而采样得到的子图及其特征则传输至GPU进行训练.然而,这种方法面临着严重的图特征数据加载瓶颈,显著降低了端到端训练性能,且图特征占用过多内存,严重限制了可训练的图规模.为了解决这些问题,我们提出了基于输入特征稀疏化的数据加载方法,显著减少CPU内存占用和跨PCIe总线传输的数据量,大幅缩短数据加载时间,加速GNN的训练,使其可以充分利用GPU计算资源.针对图特征和GNN计算特性,我们提出了适用于图特征数据的稀疏化方法,在压缩比和模型准确度之间达到平衡.我们在3个常见GNN模型和3个不同规模的数据集上进行了实验评估,包括最大的公开数据集之一MAG240M.结果表明,此方法将特征尺寸减小了一个数量级以上,并实现1.6-6.7倍的端到端训练加速,而模型准确度的降低不超过1%.此外,在仅使用4个GPU的情况下,仅需40 min就可以在MAG240M上完成GraphSAGE模型的训练并达到目标准确度.

基于图神经网络的关系抽取研究综述

在关系抽取任务中,通常利用构建依赖树或句法树来获得更深层和丰富的结构信息.图神经网络作为一种强大的图结构数据表示学习方法,可以更好地对这种复杂数据结构进行建模.本文介绍了基于图神经网络的关系抽取方法,旨在深入理解该领域的最新研究进展和趋势.首先简要介绍了图神经网络的分类和结构,然后详细阐述了基于图神经网络的关系抽取方法的核心技术和应用场景,包括句子级和文档级方法,以及实体关系联合抽取方法.并分析和比较了各个方法的优缺点和性能表现,并探讨了未来可能的研究方向和挑战.