An End-To-End Hyperbolic Deep Graph Convolutional Neural Network Framework

Graph Convolutional Neural Networks(GCNs)have been widely used in various fields due to their powerful capabilities in processing graph-structured data.However,GCNs encounter significant challenges when applied to scale-free graphs with power-law distributions,resulting in substantial distortions.Moreover,most of the existing GCN models are shallow structures,which restricts their ability to capture dependencies among distant nodes and more refined high-order node features in scale-free graphs with hierarchical structures.To more broadly and precisely apply GCNs to real-world graphs exhibiting scale-free or hierarchical structures and utilize multi-level aggregation of GCNs for capturing high-level information in local representations,we propose the Hyperbolic Deep Graph Convolutional Neural Network(HDGCNN),an end-to-end deep graph representation learning framework that can map scale-free graphs from Euclidean space to hyperbolic space.In HDGCNN,we define the fundamental operations of deep graph convolutional neural networks in hyperbolic space.Additionally,we introduce a hyperbolic feature transformation method based on identity mapping and a dense connection scheme based on a novel non-local message passing framework.In addition,we present a neighborhood aggregation method that combines initial structural featureswith hyperbolic attention coefficients.Through the above methods,HDGCNN effectively leverages both the structural features and node features of graph data,enabling enhanced exploration of non-local structural features and more refined node features in scale-free or hierarchical graphs.Experimental results demonstrate that HDGCNN achieves remarkable performance improvements over state-ofthe-art GCNs in node classification and link prediction tasks,even when utilizing low-dimensional embedding representations.Furthermore,when compared to shallow hyperbolic graph convolutional neural network models,HDGCNN exhibits notable advantages and performance enhancements.

Graph Convolutional Networks Embedding Textual Structure Information for Relation Extraction

Deep neural network-based relational extraction research has made significant progress in recent years,andit provides data support for many natural language processing downstream tasks such as building knowledgegraph,sentiment analysis and question-answering systems.However,previous studies ignored much unusedstructural information in sentences that could enhance the performance of the relation extraction task.Moreover,most existing dependency-based models utilize self-attention to distinguish the importance of context,whichhardly deals withmultiple-structure information.To efficiently leverage multiple structure information,this paperproposes a dynamic structure attention mechanism model based on textual structure information,which deeplyintegrates word embedding,named entity recognition labels,part of speech,dependency tree and dependency typeinto a graph convolutional network.Specifically,our model extracts text features of different structures from theinput sentence.Textual Structure information Graph Convolutional Networks employs the dynamic structureattention mechanism to learn multi-structure attention,effectively distinguishing important contextual features invarious structural information.In addition,multi-structure weights are carefully designed as amergingmechanismin the different structure attention to dynamically adjust the final attention.This paper combines these featuresand trains a graph convolutional network for relation extraction.We experiment on supervised relation extractiondatasets including SemEval 2010 Task 8,TACRED,TACREV,and Re-TACED,the result significantly outperformsthe previous.

Sampling Methods for Efficient Training of Graph Convolutional Networks:A Survey

Graph convolutional networks(GCNs)have received significant attention from various research fields due to the excellent performance in learning graph representations.Although GCN performs well compared with other methods,it still faces challenges.Training a GCN model for large-scale graphs in a conventional way requires high computation and storage costs.Therefore,motivated by an urgent need in terms of efficiency and scalability in training GCN,sampling methods have been proposed and achieved a significant effect.In this paper,we categorize sampling methods based on the sampling mechanisms and provide a comprehensive survey of sampling methods for efficient training of GCN.To highlight the characteristics and differences of sampling methods,we present a detailed comparison within each category and further give an overall comparative analysis for the sampling methods in all categories.Finally,we discuss some challenges and future research directions of the sampling methods.

Weighted Forwarding in Graph Convolution Networks for Recommendation Information Systems

Recommendation Information Systems(RIS)are pivotal in helping users in swiftly locating desired content from the vast amount of information available on the Internet.Graph Convolution Network(GCN)algorithms have been employed to implement the RIS efficiently.However,the GCN algorithm faces limitations in terms of performance enhancement owing to the due to the embedding value-vanishing problem that occurs during the learning process.To address this issue,we propose a Weighted Forwarding method using the GCN(WF-GCN)algorithm.The proposed method involves multiplying the embedding results with different weights for each hop layer during graph learning.By applying the WF-GCN algorithm,which adjusts weights for each hop layer before forwarding to the next,nodes with many neighbors achieve higher embedding values.This approach facilitates the learning of more hop layers within the GCN framework.The efficacy of the WF-GCN was demonstrated through its application to various datasets.In the MovieLens dataset,the implementation of WF-GCN in LightGCN resulted in significant performance improvements,with recall and NDCG increasing by up to+163.64%and+132.04%,respectively.Similarly,in the Last.FM dataset,LightGCN using WF-GCN enhanced with WF-GCN showed substantial improvements,with the recall and NDCG metrics rising by up to+174.40%and+169.95%,respectively.Furthermore,the application of WF-GCN to Self-supervised Graph Learning(SGL)and Simple Graph Contrastive Learning(SimGCL)also demonstrated notable enhancements in both recall and NDCG across these datasets.

Modeling Multi-Targets Sentiment Classification via Graph Convolutional Networks and Auxiliary Relation

Existing solutions do not work well when multi-targets coexist in a sentence.The reason is that the existing solution is usually to separate multiple targets and process them separately.If the original sentence has N target,the original sentence will be repeated for N times,and only one target will be processed each time.To some extent,this approach degenerates the fine-grained sentiment classification task into the sentence-level sentiment classification task,and the research method of processing the target separately ignores the internal relation and interaction between the targets.Based on the above considerations,we proposes to use Graph Convolutional Network(GCN)to model and process multi-targets appearing in sentences at the same time based on the positional relationship,and then to construct a graph of the sentiment relationship between targets based on the difference of the sentiment polarity between target words.In addition to the standard target-dependent sentiment classification task,an auxiliary node relation classification task is constructed.Experiments demonstrate that our model achieves new comparable performance on the benchmark datasets:SemEval-2014 Task 4,i.e.,reviews for restaurants and laptops.Furthermore,the method of dividing the target words into isolated individuals has disadvantages,and the multi-task learning model is beneficial to enhance the feature extraction ability and expression ability of the model.

Using BlazePose on Spatial Temporal Graph Convolutional Networks for Action Recognition

The ever-growing available visual data(i.e.,uploaded videos and pictures by internet users)has attracted the research community’s attention in the computer vision field.Therefore,finding efficient solutions to extract knowledge from these sources is imperative.Recently,the BlazePose system has been released for skeleton extraction from images oriented to mobile devices.With this skeleton graph representation in place,a Spatial-Temporal Graph Convolutional Network can be implemented to predict the action.We hypothesize that just by changing the skeleton input data for a different set of joints that offers more information about the action of interest,it is possible to increase the performance of the Spatial-Temporal Graph Convolutional Network for HAR tasks.Hence,in this study,we present the first implementation of the BlazePose skeleton topology upon this architecture for action recognition.Moreover,we propose the Enhanced-BlazePose topology that can achieve better results than its predecessor.Additionally,we propose different skeleton detection thresholds that can improve the accuracy performance even further.We reached a top-1 accuracy performance of 40.1%on the Kinetics dataset.For the NTU-RGB+D dataset,we achieved 87.59%and 92.1%accuracy for Cross-Subject and Cross-View evaluation criteria,respectively.

Attack Behavior Extraction Based on Heterogeneous Cyberthreat Intelligence and Graph Convolutional Networks

The continuous improvement of the cyber threat intelligence sharing mechanism provides new ideas to deal with Advanced Persistent Threats(APT).Extracting attack behaviors,i.e.,Tactics,Techniques,Procedures(TTP)from Cyber Threat Intelligence(CTI)can facilitate APT actors’profiling for an immediate response.However,it is difficult for traditional manual methods to analyze attack behaviors from cyber threat intelligence due to its heterogeneous nature.Based on the Adversarial Tactics,Techniques and Common Knowledge(ATT&CK)of threat behavior description,this paper proposes a threat behavioral knowledge extraction framework that integrates Heterogeneous Text Network(HTN)and Graph Convolutional Network(GCN)to solve this issue.It leverages the hierarchical correlation relationships of attack techniques and tactics in the ATT&CK to construct a text network of heterogeneous cyber threat intelligence.With the help of the Bidirectional EncoderRepresentation fromTransformers(BERT)pretraining model to analyze the contextual semantics of cyber threat intelligence,the task of threat behavior identification is transformed into a text classification task,which automatically extracts attack behavior in CTI,then identifies the malware and advanced threat actors.The experimental results show that F1 achieve 94.86%and 92.15%for the multi-label classification tasks of tactics and techniques.Extend the experiment to verify the method’s effectiveness in identifying the malware and threat actors in APT attacks.The F1 for malware and advanced threat actors identification task reached 98.45%and 99.48%,which are better than the benchmark model in the experiment and achieve state of the art.The model can effectivelymodel threat intelligence text data and acquire knowledge and experience migration by correlating implied features with a priori knowledge to compensate for insufficient sample data and improve the classification performance and recognition ability of threat behavior in text.

Smart Lung Tumor Prediction Using Dual Graph Convolutional Neural Network

A significant advantage of medical image processing is that it allows non-invasive exploration of internal anatomy in great detail.It is possible to create and study 3D models of anatomical structures to improve treatment outcomes,develop more effective medical devices,or arrive at a more accurate diagnosis.This paper aims to present a fused evolutionary algorithm that takes advantage of both whale optimization and bacterial foraging optimization to optimize feature extraction.The classification process was conducted with the aid of a convolu-tional neural network(CNN)with dual graphs.Evaluation of the performance of the fused model is carried out with various methods.In the initial input Com-puter Tomography(CT)image,150 images are pre-processed and segmented to identify cancerous and non-cancerous nodules.The geometrical,statistical,struc-tural,and texture features are extracted from the preprocessed segmented image using various methods such as Gray-level co-occurrence matrix(GLCM),Histo-gram-oriented gradient features(HOG),and Gray-level dependence matrix(GLDM).To select the optimal features,a novel fusion approach known as Whale-Bacterial Foraging Optimization is proposed.For the classification of lung cancer,dual graph convolutional neural networks have been employed.A com-parison of classification algorithms and optimization algorithms has been con-ducted.According to the evaluated results,the proposed fused algorithm is successful with an accuracy of 98.72%in predicting lung tumors,and it outper-forms other conventional approaches.

A malware propagation prediction model based on representation learning and graph convolutional networks

The traditional malware research is mainly based on its recognition and detection as a breakthrough point,without focusing on its propagation trends or predicting the subsequently infected nodes.The complexity of network structure,diversity of network nodes,and sparsity of data all pose difficulties in predicting propagation.This paper proposes a malware propagation prediction model based on representation learning and Graph Convolutional Networks(GCN)to address the aforementioned problems.First,to solve the problem of the inaccuracy of infection intensity calculation caused by the sparsity of node interaction behavior data in the malware propagation network,a mechanism based on a tensor to mine the infection intensity among nodes is proposed to retain the network structure information.The influence of the relationship between nodes on the infection intensity is also analyzed.Second,given the diversity and complexity of the content and structure of infected and normal nodes in the network,considering the advantages of representation learning in data feature extraction,the corresponding representation learning method is adopted for the characteristics of infection intensity among nodes.This can efficiently calculate the relationship between entities and relationships in low dimensional space to achieve the goal of low dimensional,dense,and real-valued representation learning for the characteristics of propagation spatial data.We also design a new method,Tensor2vec,to learn the potential structural features of malware propagation.Finally,considering the convolution ability of GCN for non-Euclidean data,we propose a dynamic prediction model of malware propagation based on representation learning and GCN to solve the time effectiveness problem of the malware propagation carrier.The experimental results show that the proposed model can effectively predict the behaviors of the nodes in the network and discover the influence of different characteristics of nodes on the malware propagation situation.

融合自适应周期与兴趣量因子的轻量级GCN推荐

推荐系统在成熟的数据挖掘技术推动下,已能高效地利用评分数据、行为轨迹等显隐性信息,再与复杂而先进的深度学习技术相结合,取得了很好的效果.同时,其应用需求也驱动着对基础数据的深度挖掘与利用,以及对技术要求的减负成为一个研究热点.基于此,提出一种利用GCN(graph convolutional network)方法进行深度信息融合的轻量级推荐模型LG_APIF.该模型结合行为记忆,通过艾宾浩斯遗忘曲线模拟用户兴趣变化过程,采用线性回归等相对轻量的传统方法挖掘项目的自适应周期等深度信息;分析用户当前的兴趣分布,计算项目的兴趣量,以获取用户的潜在兴趣类型;构建用户-类型-项目三元组的图结构,并结合减负后的GCN技术来生成最终的项目推荐列表.实验验证所提方法的有效性,通过与8个经典模型在Last.fm,Douban,Yelp,MovieLens数据集中的对比,表明该方法在Precision,Recall及NDCG指标上都得到良好改善,其中,Precision平均提升2.11%,Recall平均提升1.01%,NDCG平均提升1.48%.

GCN引导模型视点的光学遥感道路提取网络

在光学遥感图像中,道路易受遮挡物、铺装材料以及周围环境等多重因素的影响,导致其特征模糊不清。然而,现有道路提取方法即使增强其特征感知能力,仍在特征模糊区域存在大量误判。为解决上述问题,本文提出基于GCN引导模型视点的道路提取网络(RGGVNet)。RGGVNet采用编解码结构,并设计基于GCN的视点引导模块(GVPG)在编解码器的连接处反复引导模型视点,从而增强对特征模糊区域的关注。GVPG利用GCN信息传播过程具有平均特征权重的特性,将特征图中不同区域道路显著性水平作为拉普拉斯矩阵,参与到GCN信息传播从而实现引导模型视点。同时,提出密集引导视点策略(DGVS),采用密集连接的方式将编码器、GVPG和解码器相互连接,确保有效引导模型视点的同时缓解优化困难。在解码阶段设计多分辨率特征融合(MRFF)模块,最小化不同尺度道路特征在特征融合和上采样过程中的信息偏移和损失。在两个公开遥感道路数据集中,本文方法IoU分别达到65.84%和69.36%,F1-score分别达到79.40%和81.90%。从定量和定性两方面实验结果可以看出,本文所提方法性能优于其他主流方法。

基于DCGCN模型的海上风电场超短期功率预测

图卷积网络(GCN)具有很强的数据关联挖掘能力,近年来在风电功率预测领域获得了广泛关注。然而,传统的基于GCN模型的超短期风电功率预测难以同时处理影响风电功率的两大核心因素(风速与机组状态信息)的双模态问题,基于此,提出了一种基于双通道图卷积网络(DCGCN)的海上风电场超短期功率预测模型。首先,建立以理论功率曲线为基准的机组状态指标模型,定量表征机组状态变化对其发电能力的影响;其次,构建海上风电场图拓扑,建立基于风速和状态邻接矩阵的风电场各机组捕获的风速与机组状态信息的关联关系模型;最后,建立基于DCGCN的风电场超短期功率预测方法。算例结果表明,所提模型有助于提高风电场功率预测模型的训练效率和预测精度。

一种基于GCN的光伏短期出力预测方法研究

为提高光伏短期出力预测精确问题,提出一种基于图卷积神经网络(GCN)的光伏短期出力预测方法。首先,构建考虑多气象影响因素的光伏短期出力模型,开展光伏出力影响因素和出力特性分析。其次,对光伏出力历史时序数据进行图形化转换和数据重构,构建邻接矩阵并提取光伏短期出力图形化特征数据。在多时间尺度场景下,建立基于GCN的光伏出力预测模型,并与基于长短期记忆网络(LSTM)、反向传播网络(BP)、图注意力模型(GAT)等算法的预测模型做比对分析。最后,以某地区光伏出力实测数据开展仿真验证研究,仿真结果表明所提方法具有良好的预测效果。

融合RoBERTa-GCN-Attention的隐喻识别与情感分类模型

在隐喻识别与隐喻情感分类任务的联合研究中,现有多任务学习模型存在对隐喻语料中的上下文语义信息和句法结构信息提取不够准确,并且缺乏对粗细两种粒度信息同时捕捉的问题.针对第1个问题,首先改进了传统的RoBERTa模型,在原有的自注意力机制中引入上下文信息,以此提取上下文中重要的隐喻语义特征;其次在句法依存树上使用图卷积网络提取隐喻句中的句法结构信息.针对第2个问题,使用双层注意力机制,分别聚焦于单词和句子层面中对隐喻识别和情感分类有贡献的特征信息.在两类任务6个数据集上的对比实验结果表明,该模型相比基线模型性能均有提升.

基于物联网和GCNN-LSTM的河流水文预测方法

针对河流水文存在预测精度不高的问题,利用物联网技术设计了分布式的降雨和水文信息自动采集系统,并提出了一种基于图卷积神经网络和长短期记忆网络模型对河流水位和径流量进行预测的方法;首先通过分析确定了影响河流水文的主要因素,将流域范围内的降雨量信息组成网格化的二维图形矩阵;然后提出了GCNN-LSTM预测模型,将含有降雨信息的二维图形矩阵作为网络模型的输入,获取该流域内降雨与水文变化的时空分布特征;最后采用所提出的GCNN-LSTM预测模型对河南省周口市段颍河的历史水文数据进行训练,再利用训练后的网络对测试集数据进行预测,得到了较高精度的径流量和水位结果,径流量预测结果的RMSE、MAPE和MAE分别仅为17.09 m^(3)/s、1.68%和8.57 m^(3)/s,水位预测结果的RMSE、MAPE和MAE分别仅为0.32 m、0.65%和0.29 m,与其他几种预测方法相比表现出了优越性,对科学合理利用水资源和防洪减灾具有重要意义。

多视角融合的时空动态GCN城市交通流量预测

城市交通流量预测是构建绿色低碳、安全高效的智能交通系统的重要组成部分.时空图神经网络由于具有强大的时空数据表征能力,被广泛应用于城市交通流量预测.当前,时空图神经网络在城市交通流量预测中仍存在以下两方面局限性:1)直接构建静态路网拓扑图对城市空间相关性进行表示,忽略了节点的动态交通模式,难以表达节点流量之间的时序相似性,无法捕获路网节点之间在时序上的动态关联;2)只考虑路网节点的局部空间相关性,忽略节点的全局空间相关性,无法建模交通路网中局部区域和全局空间之间的依赖关系.为打破上述局限性,提出了一种多视角融合的时空动态图卷积模型用于预测交通流量:首先,从静态空间拓扑和动态流量模式视角出发,构建路网空间结构图和动态流量关联图,并使用动态图卷积学习节点在两种视角下的特征,全面捕获城市路网中多元的空间相关性;其次,从局部视角和全局视角出发,计算路网的全局表示,将全局特征与局部特征融合,增强路网节点特征的表现力,发掘城市交通流量的整体结构特征;接下来,设计了局部卷积多头自注意力机制来获取交通数据的动态时间相关性,实现在多种时间窗口下的准确流量预测;最后,在4种真实交通数据上的实验结果,证明了该模型的有效性和准确性.

Micro-expression recognition algorithm based on graph convolutional network and Transformer model

Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.

基于Graph Transformer的半监督异配图表示学习模型

现有的图卷积网络(GCN)模型基于同配性假设,无法直接应用于异配图的表示学习,且许多异配图表示学习的研究工作受消息传递机制的限制,导致节点特征混淆和特征过度挤压而出现过平滑问题。针对这些问题,提出一种基于Graph Transformer的半监督异配图表示学习模型HPGT(HeteroPhilic Graph Transformer)。首先,使用度连接概率矩阵采样节点的路径邻域,再通过自注意力机制自适应地聚合路径上的节点异配连接模式,编码得到节点的结构信息,用节点的原始属性信息和结构信息构建Transformer层的自注意力模块;其次,将每个节点自身的隐层表示与它的邻域节点的隐层表示分离更新以避免节点通过自注意力模块聚合过量的自身信息,再把每个节点表示与它的邻域表示连接,得到单个Transformer层的输出,另外,将所有的Transformer层的输出跳连到最终的节点隐层表示以防止中间层信息丢失;最后,使用线性层和Softmax层将节点的隐层表示映射到节点的预测标签。实验结果表明,与无结构编码(SE)的模型相比,基于度连接概率的SE能为Transformer层的自注意力模块提供有效的偏差信息,HPGT平均准确率提升0.99%~11.98%;与对比模型相比,在异配数据集(Texas、Cornell、Wisconsin和Actor)上,模型节点分类准确率提升0.21%~1.69%,在同配数据集(Cora、CiteSeer和PubMed)上,节点分类准确率分别达到了0.8379、0.7467和0.8862。以上结果验证了HPGT具有较强的异配图表示学习能力,尤其适用于强异配图节点分类任务。

Deep convolutional adversarial graph autoencoder using positive pointwise mutual information for graph embedding

Graph embedding aims to map the high-dimensional nodes to a low-dimensional space and learns the graph relationship from its latent representations.Most existing graph embedding methods focus on the topological structure of graph data,but ignore the semantic information of graph data,which results in the unsatisfied performance in practical applications.To overcome the problem,this paper proposes a novel deep convolutional adversarial graph autoencoder(GAE)model.To embed the semantic information between nodes in the graph data,the random walk strategy is first used to construct the positive pointwise mutual information(PPMI)matrix,then,graph convolutional net-work(GCN)is employed to encode the PPMI matrix and node content into the latent representation.Finally,the learned latent representation is used to reconstruct the topological structure of the graph data by decoder.Furthermore,the deep convolutional adversarial training algorithm is introduced to make the learned latent representation conform to the prior distribution better.The state-of-the-art experimental results on the graph data validate the effectiveness of the proposed model in the link prediction,node clustering and graph visualization tasks for three standard datasets,Cora,Citeseer and Pubmed.

Skeleton Split Strategies for Spatial Temporal Graph Convolution Networks

Action recognition has been recognized as an activity in which individuals’behaviour can be observed.Assembling profiles of regular activities such as activities of daily living can support identifying trends in the data during critical events.A skeleton representation of the human body has been proven to be effective for this task.The skeletons are presented in graphs form-like.However,the topology of a graph is not structured like Euclideanbased data.Therefore,a new set of methods to perform the convolution operation upon the skeleton graph is proposed.Our proposal is based on the Spatial Temporal-Graph Convolutional Network(ST-GCN)framework.In this study,we proposed an improved set of label mapping methods for the ST-GCN framework.We introduce three split techniques(full distance split,connection split,and index split)as an alternative approach for the convolution operation.The experiments presented in this study have been trained using two benchmark datasets:NTU-RGB+D and Kinetics to evaluate the performance.Our results indicate that our split techniques outperform the previous partition strategies and aremore stable during training without using the edge importance weighting additional training parameter.Therefore,our proposal can provide a more realistic solution for real-time applications centred on daily living recognition systems activities for indoor environments.