Semi-Supervised Graph Learning for Brain Disease Identification

Using resting-state functional magnetic resonance imaging (fMRI) technology to assist in identifying brain diseases has great potential. In the identification of brain diseases, graph-based models have been widely used, where graph represents the similarity between patients or brain regions of interest. In these models, constructing high-quality graphs is of paramount importance. Researchers have proposed various methods for constructing graphs from different perspectives, among which the simplest and most popular one is Pearson Correlation (PC). Although existing methods have achieved significant results, these graphs are usually fixed once they are constructed, and are generally operated separately from downstream task. Such a separation may result in neither the constructed graph nor the extracted features being ideal. To solve this problem, we use the graph-optimized locality preserving projection algorithm to extract features and the population graph simultaneously, aiming in higher identification accuracy through a task-dependent automatic optimization of the graph. At the same time, we incorporate supervised information to enable more flexible modelling. Specifically, the proposed method first uses PC to construct graph as the initial feature for each subject. Then, the projection matrix and graph are iteratively optimized through graph-optimization locality preserving projections based on semi-supervised learning, which fully employs the knowledge in various transformation spaces. Finally, the obtained projection matrix is applied to construct the subject-level graph and perform classification using support vector machines. To verify the effectiveness of the proposed method, we conduct experiments to identify subjects with mild cognitive impairment (MCI) and Autism spectrum disorder (ASD) from normal controls (NCs), and the results showed that the classification performance of our method is better than that of the baseline method.

A Novel Graph Structure Learning Based Semi-Supervised Framework for Anomaly Identification in Fluctuating IoT Environment

With the rapid development of Internet of Things(IoT)technology,IoT systems have been widely applied in health-care,transportation,home,and other fields.However,with the continuous expansion of the scale and increasing complexity of IoT systems,the stability and security issues of IoT systems have become increasingly prominent.Thus,it is crucial to detect anomalies in the collected IoT time series from various sensors.Recently,deep learning models have been leveraged for IoT anomaly detection.However,owing to the challenges associated with data labeling,most IoT anomaly detection methods resort to unsupervised learning techniques.Nevertheless,the absence of accurate abnormal information in unsupervised learning methods limits their performance.To address these problems,we propose AS-GCN-MTM,an adaptive structural Graph Convolutional Networks(GCN)-based framework using a mean-teacher mechanism(AS-GCN-MTM)for anomaly identification.It performs better than unsupervised methods using only a small amount of labeled data.Mean Teachers is an effective semi-supervised learning method that utilizes unlabeled data for training to improve the generalization ability and performance of the model.However,the dependencies between data are often unknown in time series data.To solve this problem,we designed a graph structure adaptive learning layer based on neural networks,which can automatically learn the graph structure from time series data.It not only better captures the relationships between nodes but also enhances the model’s performance by augmenting key data.Experiments have demonstrated that our method improves the baseline model with the highest F1 value by 10.4%,36.1%,and 5.6%,respectively,on three real datasets with a 10%data labeling rate.

Fault Location and Classification for Distribution Systems Based on Deep Graph Learning Methods

Accurate and timely fault diagnosis is of great significance for the safe operation and power supply reliability of distribution systems.However,traditional intelligent methods limit the use of the physical structures and data information of power networks.To this end,this study proposes a fault diagnostic model for distribution systems based on deep graph learning.This model considers the physical structure of the power network as a significant constraint during model training,which endows the model with stronger information perception to resist abnormal data input and unknown application conditions.In addition,a special spatiotemporal convolutional block is utilized to enhance the waveform feature extraction ability.This enables the proposed fault diagnostic model to be more effective in dealing with both fault waveform changes and the spatial effects of faults.In addition,a multi-task learning framework is constructed for fault location and fault type analysis,which improves the performance and generalization ability of the model.The IEEE 33-bus and IEEE 37-bus test systems are modeled to verify the effectiveness of the proposed fault diagnostic model.Finally,different fault conditions,topological changes,and interference factors are considered to evaluate the anti-interference and generalization performance of the proposed model.Experimental results demonstrate that the proposed model outperforms other state-of-the-art methods.

PGSLM:Edge-Enabled Probabilistic Graph Structure Learning Model for Traffic Forecasting in Internet of Vehicles

With the rapid development of the 5G communications,the edge intelligence enables Internet of Vehicles(IoV)to provide traffic forecasting to alleviate traffic congestion and improve quality of experience of users simultaneously.To enhance the forecasting performance,a novel edge-enabled probabilistic graph structure learning model(PGSLM)is proposed,which learns the graph structure and parameters by the edge sensing information and discrete probability distribution on the edges of the traffic road network.To obtain the spatio-temporal dependencies of traffic data,the learned dynamic graphs are combined with a predefined static graph to generate the graph convolution part of the recurrent graph convolution module.During the training process,a new graph training loss is introduced,which is composed of the K nearest neighbor(KNN)graph constructed by the traffic feature tensors and the graph structure.Detailed experimental results show that,compared with existing models,the proposed PGSLM improves the traffic prediction performance in terms of average absolute error and root mean square error in IoV.

CoLM^(2)S:Contrastive self‐supervised learning on attributed multiplex graph network with multi‐scale information

Contrastive self‐supervised representation learning on attributed graph networks with Graph Neural Networks has attracted considerable research interest recently.However,there are still two challenges.First,most of the real‐word system are multiple relations,where entities are linked by different types of relations,and each relation is a view of the graph network.Second,the rich multi‐scale information(structure‐level and feature‐level)of the graph network can be seen as self‐supervised signals,which are not fully exploited.A novel contrastive self‐supervised representation learning framework on attributed multiplex graph networks with multi‐scale(named CoLM^(2)S)information is presented in this study.It mainly contains two components:intra‐relation contrast learning and interrelation contrastive learning.Specifically,the contrastive self‐supervised representation learning framework on attributed single‐layer graph networks with multi‐scale information(CoLMS)framework with the graph convolutional network as encoder to capture the intra‐relation information with multi‐scale structure‐level and feature‐level selfsupervised signals is introduced first.The structure‐level information includes the edge structure and sub‐graph structure,and the feature‐level information represents the output of different graph convolutional layer.Second,according to the consensus assumption among inter‐relations,the CoLM^(2)S framework is proposed to jointly learn various graph relations in attributed multiplex graph network to achieve global consensus node embedding.The proposed method can fully distil the graph information.Extensive experiments on unsupervised node clustering and graph visualisation tasks demonstrate the effectiveness of our methods,and it outperforms existing competitive baselines.

Heterophilic Graph Neural Network Based on Spatial and Frequency Domain Adaptive Embedding Mechanism

Graph Neural Networks(GNNs)play a significant role in tasks related to homophilic graphs.Traditional GNNs,based on the assumption of homophily,employ low-pass filters for neighboring nodes to achieve information aggregation and embedding.However,in heterophilic graphs,nodes from different categories often establish connections,while nodes of the same category are located further apart in the graph topology.This characteristic poses challenges to traditional GNNs,leading to issues of“distant node modeling deficiency”and“failure of the homophily assumption”.In response,this paper introduces the Spatial-Frequency domain Adaptive Heterophilic Graph Neural Networks(SFA-HGNN),which integrates adaptive embedding mechanisms for both spatial and frequency domains to address the aforementioned issues.Specifically,for the first problem,we propose the“Distant Spatial Embedding Module”,aiming to select and aggregate distant nodes through high-order randomwalk transition probabilities to enhance modeling capabilities.For the second issue,we design the“Proximal Frequency Domain Embedding Module”,constructing adaptive filters to separate high and low-frequency signals of nodes,and introduce frequency-domain guided attention mechanisms to fuse the relevant information,thereby reducing the noise introduced by the failure of the homophily assumption.We deploy the SFA-HGNN on six publicly available heterophilic networks,achieving state-of-the-art results in four of them.Furthermore,we elaborate on the hyperparameter selection mechanism and validate the performance of each module through experimentation,demonstrating a positive correlation between“node structural similarity”,“node attribute vector similarity”,and“node homophily”in heterophilic networks.

Weighted Forwarding in Graph Convolution Networks for Recommendation Information Systems

Recommendation Information Systems(RIS)are pivotal in helping users in swiftly locating desired content from the vast amount of information available on the Internet.Graph Convolution Network(GCN)algorithms have been employed to implement the RIS efficiently.However,the GCN algorithm faces limitations in terms of performance enhancement owing to the due to the embedding value-vanishing problem that occurs during the learning process.To address this issue,we propose a Weighted Forwarding method using the GCN(WF-GCN)algorithm.The proposed method involves multiplying the embedding results with different weights for each hop layer during graph learning.By applying the WF-GCN algorithm,which adjusts weights for each hop layer before forwarding to the next,nodes with many neighbors achieve higher embedding values.This approach facilitates the learning of more hop layers within the GCN framework.The efficacy of the WF-GCN was demonstrated through its application to various datasets.In the MovieLens dataset,the implementation of WF-GCN in LightGCN resulted in significant performance improvements,with recall and NDCG increasing by up to+163.64%and+132.04%,respectively.Similarly,in the Last.FM dataset,LightGCN using WF-GCN enhanced with WF-GCN showed substantial improvements,with the recall and NDCG metrics rising by up to+174.40%and+169.95%,respectively.Furthermore,the application of WF-GCN to Self-supervised Graph Learning(SGL)and Simple Graph Contrastive Learning(SimGCL)also demonstrated notable enhancements in both recall and NDCG across these datasets.

Review on graph learning for dimensionality reduction of hyperspectral image

Graph learning is an effective manner to analyze the intrinsic properties of data.It has been widely used in the fields of dimensionality reduction and classification for data.In this paper,we focus on the graph learning-based dimensionality reduction for a hyperspectral image.Firstly,we review the development of graph learning and its application in a hyperspectral image.Then,we mainly discuss several representative graph methods including two manifold learning methods,two sparse graph learning methods,and two hypergraph learning methods.For manifold learning,we analyze neighborhood preserving embedding and locality preserving projections which are two classic manifold learning methods and can be transformed into the form of a graph.For sparse graph,we introduce sparsity preserving graph embedding and sparse graph-based discriminant analysis which can adaptively reveal data structure to construct a graph.For hypergraph learning,we review binary hypergraph and discriminant hyper-Laplacian projection which can represent the high-order relationship of data.

Graph Laplacian Matrix Learning from Smooth Time-Vertex Signal

In this paper,we focus on inferring graph Laplacian matrix from the spatiotemporal signal which is defined as“time-vertex signal”.To realize this,we first represent the signals on a joint graph which is the Cartesian product graph of the time-and vertex-graphs.By assuming the signals follow a Gaussian prior distribution on the joint graph,a meaningful representation that promotes the smoothness property of the joint graph signal is derived.Furthermore,by decoupling the joint graph,the graph learning framework is formulated as a joint optimization problem which includes signal denoising,timeand vertex-graphs learning together.Specifically,two algorithms are proposed to solve the optimization problem,where the discrete second-order difference operator with reversed sign(DSODO)in the time domain is used as the time-graph Laplacian operator to recover the signal and infer a vertex-graph in the first algorithm,and the time-graph,as well as the vertex-graph,is estimated by the other algorithm.Experiments on both synthetic and real-world datasets demonstrate that the proposed algorithms can effectively infer meaningful time-and vertex-graphs from noisy and incomplete data.

Future Event Prediction Based on Temporal Knowledge Graph Embedding

Accurate prediction of future events brings great benefits and reduces losses for society in many domains,such as civil unrest,pandemics,and crimes.Knowledge graph is a general language for describing and modeling complex systems.Different types of events continually occur,which are often related to historical and concurrent events.In this paper,we formalize the future event prediction as a temporal knowledge graph reasoning problem.Most existing studies either conduct reasoning on static knowledge graphs or assume knowledges graphs of all timestamps are available during the training process.As a result,they cannot effectively reason over temporal knowledge graphs and predict events happening in the future.To address this problem,some recent works learn to infer future events based on historical eventbased temporal knowledge graphs.However,these methods do not comprehensively consider the latent patterns and influences behind historical events and concurrent events simultaneously.This paper proposes a new graph representation learning model,namely Recurrent Event Graph ATtention Network(RE-GAT),based on a novel historical and concurrent events attention-aware mechanism by modeling the event knowledge graph sequence recurrently.More specifically,our RE-GAT uses an attention-based historical events embedding module to encode past events,and employs an attention-based concurrent events embedding module to model the associations of events at the same timestamp.A translation-based decoder module and a learning objective are developed to optimize the embeddings of entities and relations.We evaluate our proposed method on four benchmark datasets.Extensive experimental results demonstrate the superiority of our RE-GAT model comparing to various base-lines,which proves that our method can more accurately predict what events are going to happen.

A graph deep learning method for landslide displacement prediction based on global navigation satellite system positioning

The accurate prediction of displacement is crucial for landslide deformation monitoring and early warning.This study focuses on a landslide in Wenzhou Belt Highway and proposes a novel multivariate landslide displacement prediction method that relies on graph deep learning and Global Navigation Satellite System(GNSS)positioning.First model the graph structure of the monitoring system based on the engineering positions of the GNSS monitoring points and build the adjacent matrix of graph nodes.Then construct the historical and predicted time series feature matrixes using the processed temporal data including GNSS displacement,rainfall,groundwater table and soil moisture content and the graph structure.Last introduce the state-of-the-art graph deep learning GTS(Graph for Time Series)model to improve the accuracy and reliability of landslide displacement prediction which utilizes the temporal-spatial dependency of the monitoring system.This approach outperforms previous studies that only learned temporal features from a single monitoring point and maximally weighs the prediction performance and the priori graph of the monitoring system.The proposed method performs better than SVM,XGBoost,LSTM and DCRNN models in terms of RMSE(1.35 mm),MAE(1.14 mm)and MAPE(0.25)evaluation metrics,which is provided to be effective in future landslide failure early warning.

False Negative Sample Detection for Graph Contrastive Learning

Recently,self-supervised learning has shown great potential in Graph Neural Networks (GNNs) through contrastive learning,which aims to learn discriminative features for each node without label information. The key to graph contrastive learning is data augmentation. The anchor node regards its augmented samples as positive samples,and the rest of the samples are regarded as negative samples,some of which may be positive samples. We call these mislabeled samples as “false negative” samples,which will seriously affect the final learning effect. Since such semantically similar samples are ubiquitous in the graph,the problem of false negative samples is very significant. To address this issue,the paper proposes a novel model,False negative sample Detection for Graph Contrastive Learning (FD4GCL),which uses attribute and structure-aware to detect false negative samples. Experimental results on seven datasets show that FD4GCL outperforms the state-of-the-art baselines and even exceeds several supervised methods.

Graph representation learning-based residential electricity behavior identification and energy management

It is important to achieve an efficient home energy management system(HEMS)because of its role in promoting energy saving and emission reduction for end-users.Two critical issues in an efficient HEMS are identification of user behavior and energy management strategy.However,current HEMS methods usually assume perfect knowledge of user behavior or ignore the strong correlations of usage habits with different applications.This can lead to an insuffi-cient description of behavior and suboptimal management strategy.To address these gaps,this paper proposes non-intrusive load monitoring(NILM)assisted graph reinforcement learning(GRL)for intelligent HEMS decision making.First,a behavior correlation graph incorporating NILM is introduced to represent the energy consumption behavior of users and a multi-label classification model is used to monitor the loads.Thus,efficient identification of user behavior and description of state transition can be achieved.Second,based on the online updating of the behavior correlation graph,a GRL model is proposed to extract information contained in the graph.Thus,reliable strategy under uncer-tainty of environment and behavior is available.Finally,the experimental results on several datasets verify the effec-tiveness of the proposed model.

NGAT:attention in breadth and depth exploration for semi-supervised graph representation learning

Recently,graph neural networks(GNNs)have achieved remarkable performance in representation learning on graph-structured data.However,as the number of network layers increases,GNNs based on the neighborhood aggregation strategy deteriorate due to the problem of oversmoothing,which is the major bottleneck for applying GNNs to real-world graphs.Many efforts have been made to improve the process of feature information aggregation from directly connected nodes,i.e.,breadth exploration.However,these models perform the best only in the case of three or fewer layers,and the performance drops rapidly for deep layers.To alleviate oversmoothing,we propose a nested graph attention network(NGAT),which can work in a semi-supervised manner.In addition to breadth exploration,a k-layer NGAT uses a layer-wise aggregation strategy guided by the attention mechanism to selectively leverage feature information from the k;-order neighborhood,i.e.,depth exploration.Even with a 10-layer or deeper architecture,NGAT can balance the need for preserving the locality(including root node features and the local structure)and aggregating the information from a large neighborhood.In a number of experiments on standard node classification tasks,NGAT outperforms other novel models and achieves state-of-the-art performance.

Imbalanced fault diagnosis of rotating machinery using autoencoder-based SuperGraph feature learning

Existing fault diagnosis methods usually assume that there are balanced training data for every machine health state.However,the collection of fault signals is very difficult and expensive,resulting in the problem of imbalanced training dataset.It will degrade the performance of fault diagnosis methods significantly.To address this problem,an imbalanced fault diagnosis of rotating machinery using autoencoder-based SuperGraph feature learning is proposed in this paper.Unsupervised autoencoder is firstly used to compress every monitoring signal into a low-dimensional vector as the node attribute in the SuperGraph.And the edge connections in the graph depend on the relationship between signals.On the basis,graph convolution is performed on the constructed SuperGraph to achieve imbalanced training dataset fault diagnosis for rotating machinery.Comprehensive experiments are conducted on a benchmarking publicized dataset and a practical experimental platform,and the results show that the proposed method can effectively achieve rotating machinery fault diagnosis towards imbalanced training dataset through graph feature learning.

Exploring financially constrained small- and medium-sized enterprises based on a multi-relation translational graph attention network

Financing needs exploration(FNE),which exploresfinancially constrained small-and medium-sized enterprises(SMEs),has become increasingly important in industry forfinancial institutions to facilitate SMEs’development.In this paper,wefirst perform an insightful exploratory analysis to exploit the transfer phenomenon offinancing needs among SMEs,which motivates us to fully exploit the multi-relation enterprise social network for boosting the effectiveness of FNE.The main challenge lies in modeling two kinds of heterogeneity,i.e.,transfer heterogeneity and SMEs’behavior heterogeneity,under different relation types simultaneously.To address these challenges,we propose a graph neural network named Multi-relation tRanslatIonal GrapH a Ttention network(M-RIGHT),which not only models the transfer heterogeneity offinancing needs along different relation types based on a novel entity–relation composition operator but also enables heterogeneous SMEs’representations based on a translation mechanism on relational hyperplanes to distinguish SMEs’heterogeneous behaviors under different relation types.Extensive experiments on two large-scale real-world datasets demonstrate M-RIGHT’s superiority over the state-of-the-art methods in the FNE task.

Limits of Depth: Over-Smoothing and Over-Squashing in GNNs

Graph Neural Networks(GNNs)have become a widely used tool for learning and analyzing data on graph structures,largely due to their ability to preserve graph structure and properties via graph representation learning.However,the effect of depth on the performance of GNNs,particularly isotropic and anisotropic models,remains an active area of research.This study presents a comprehensive exploration of the impact of depth on GNNs,with a focus on the phenomena of over-smoothing and the bottleneck effect in deep graph neural networks.Our research investigates the tradeoff between depth and performance,revealing that increasing depth can lead to over-smoothing and a decrease in performance due to the bottleneck effect.We also examine the impact of node degrees on classification accuracy,finding that nodes with low degrees can pose challenges for accurate classification.Our experiments use several benchmark datasets and a range of evaluation metrics to compare isotropic and anisotropic GNNs of varying depths,also explore the scalability of these models.Our findings provide valuable insights into the design of deep GNNs and offer potential avenues for future research to improve their performance.

Learning from the crowd:Road infrastructure monitoring system

The condition of the road infrastructure has severe impacts on the road safety, driving comfort, and on the rolling resistance. Therefore, the road infrastructure must be moni- tored comprehensively and in regular intervals to identify damaged road segments and road hazards. Methods have been developed to comprehensively and automatically digitize the road infrastructure and estimate the road quality, which are based on vehicle sensors and a supervised machine learning classification. Since different types of vehicles have various suspension systems with different response functions, one classifier cannot be taken over to other vehicles. Usually, a high amount of time is needed to acquire training data for each individual vehicle and classifier. To address this problem, the methods to collect training data automatically for new vehicles based on the comparison of trajectories of untrained and trained vehicles have been developed. The results show that the method based on a k-dimensional tree and Euclidean distance performs best and is robust in transferring the information of the road surface from one vehicle to another. Furthermore, this method offers the possibility to merge the output and road infrastructure information from multiple vehicles to enable a more robust and precise prediction of the ground truth.

Source-Aware Embedding Training on Heterogeneous Information Networks

Heterogeneous information networks(HINs)have been extensively applied to real-world tasks,such as recommendation systems,social networks,and citation networks.While existing HIN representation learning methods can effectively learn the semantic and structural features in the network,little awareness was given to the distribution discrepancy of subgraphs within a single HIN.However,we find that ignoring such distribution discrepancy among subgraphs from multiple sources would hinder the effectiveness of graph embedding learning algorithms.This motivates us to propose SUMSHINE(Scalable Unsupervised Multi-Source Heterogeneous Information Network Embedding)-a scalable unsupervised framework to align the embedding distributions among multiple sources of an HiN.Experimental results on real-world datasets in a variety of downstream tasks validate the performance of our method over the state-of-the-art heterogeneous information network embedding algorithms.

Combating emerging financial risks in the big data era:A perspective review

financial services:for example,GPS and Bluetooth inspire location-based services,and search and web technologies motivate online shopping,reviews,and payments.These business services have become more connected than ever,and as a result,financial frauds have become a significant challenge.Therefore,combating financial risks in the big data era requires breaking the borders of traditional data,algorithms,and systems.An increasing number of studies have addressed these challenges and proposed new methods for risk detection,assessment,and forecasting.As a key contribution,we categorize these works in a rational framework:first,we identify the data that can be used to identify risks.We then discuss how big data can be combined with the emerging tools to effectively learn or analyze financial risk.Finally,we highlight the effectiveness of these methods in real-world applications.Furthermore,we stress on the importance of utilizing multi-channel information,graphs,and networks of long-range dependence for the effective identification of financial risks.We conclude our survey with a discussion on the new challenges faced by the financial sector,namely,deep fake technology,adversaries,causal and interpretable inference,privacy protection,and microsimulations.