An End-To-End Hyperbolic Deep Graph Convolutional Neural Network Framework

Graph Convolutional Neural Networks(GCNs)have been widely used in various fields due to their powerful capabilities in processing graph-structured data.However,GCNs encounter significant challenges when applied to scale-free graphs with power-law distributions,resulting in substantial distortions.Moreover,most of the existing GCN models are shallow structures,which restricts their ability to capture dependencies among distant nodes and more refined high-order node features in scale-free graphs with hierarchical structures.To more broadly and precisely apply GCNs to real-world graphs exhibiting scale-free or hierarchical structures and utilize multi-level aggregation of GCNs for capturing high-level information in local representations,we propose the Hyperbolic Deep Graph Convolutional Neural Network(HDGCNN),an end-to-end deep graph representation learning framework that can map scale-free graphs from Euclidean space to hyperbolic space.In HDGCNN,we define the fundamental operations of deep graph convolutional neural networks in hyperbolic space.Additionally,we introduce a hyperbolic feature transformation method based on identity mapping and a dense connection scheme based on a novel non-local message passing framework.In addition,we present a neighborhood aggregation method that combines initial structural featureswith hyperbolic attention coefficients.Through the above methods,HDGCNN effectively leverages both the structural features and node features of graph data,enabling enhanced exploration of non-local structural features and more refined node features in scale-free or hierarchical graphs.Experimental results demonstrate that HDGCNN achieves remarkable performance improvements over state-ofthe-art GCNs in node classification and link prediction tasks,even when utilizing low-dimensional embedding representations.Furthermore,when compared to shallow hyperbolic graph convolutional neural network models,HDGCNN exhibits notable advantages and performance enhancements.

Smart Lung Tumor Prediction Using Dual Graph Convolutional Neural Network

A significant advantage of medical image processing is that it allows non-invasive exploration of internal anatomy in great detail.It is possible to create and study 3D models of anatomical structures to improve treatment outcomes,develop more effective medical devices,or arrive at a more accurate diagnosis.This paper aims to present a fused evolutionary algorithm that takes advantage of both whale optimization and bacterial foraging optimization to optimize feature extraction.The classification process was conducted with the aid of a convolu-tional neural network(CNN)with dual graphs.Evaluation of the performance of the fused model is carried out with various methods.In the initial input Com-puter Tomography(CT)image,150 images are pre-processed and segmented to identify cancerous and non-cancerous nodules.The geometrical,statistical,struc-tural,and texture features are extracted from the preprocessed segmented image using various methods such as Gray-level co-occurrence matrix(GLCM),Histo-gram-oriented gradient features(HOG),and Gray-level dependence matrix(GLDM).To select the optimal features,a novel fusion approach known as Whale-Bacterial Foraging Optimization is proposed.For the classification of lung cancer,dual graph convolutional neural networks have been employed.A com-parison of classification algorithms and optimization algorithms has been con-ducted.According to the evaluated results,the proposed fused algorithm is successful with an accuracy of 98.72%in predicting lung tumors,and it outper-forms other conventional approaches.

Graph Convolutional Networks Embedding Textual Structure Information for Relation Extraction

Deep neural network-based relational extraction research has made significant progress in recent years,andit provides data support for many natural language processing downstream tasks such as building knowledgegraph,sentiment analysis and question-answering systems.However,previous studies ignored much unusedstructural information in sentences that could enhance the performance of the relation extraction task.Moreover,most existing dependency-based models utilize self-attention to distinguish the importance of context,whichhardly deals withmultiple-structure information.To efficiently leverage multiple structure information,this paperproposes a dynamic structure attention mechanism model based on textual structure information,which deeplyintegrates word embedding,named entity recognition labels,part of speech,dependency tree and dependency typeinto a graph convolutional network.Specifically,our model extracts text features of different structures from theinput sentence.Textual Structure information Graph Convolutional Networks employs the dynamic structureattention mechanism to learn multi-structure attention,effectively distinguishing important contextual features invarious structural information.In addition,multi-structure weights are carefully designed as amergingmechanismin the different structure attention to dynamically adjust the final attention.This paper combines these featuresand trains a graph convolutional network for relation extraction.We experiment on supervised relation extractiondatasets including SemEval 2010 Task 8,TACRED,TACREV,and Re-TACED,the result significantly outperformsthe previous.

Detecting large-scale underwater cracks based on remote operated vehicle and graph convolutional neural network

It is of great significance to quickly detect underwater cracks as they can seriously threaten the safety of underwater structures.Research to date has mainly focused on the detection of above-water-level cracks and hasn’t considered the large scale cracks.In this paper,a large-scale underwater crack examination method is proposed based on image stitching and segmentation.In addition,a purpose of this paper is to design a new convolution method to segment underwater images.An improved As-Projective-As-Possible(APAP)algorithm was designed to extract and stitch keyframes from videos.The graph convolutional neural network(GCN)was used to segment the stitched image.The GCN’s m-IOU is 24.02%higher than Fully convolutional networks(FCN),proving that GCN has great potential of application in image segmentation and underwater image processing.The result shows that the improved APAP algorithm and GCN can adapt to complex underwater environments and perform well in different study areas.

Skeleton Split Strategies for Spatial Temporal Graph Convolution Networks

Action recognition has been recognized as an activity in which individuals’behaviour can be observed.Assembling profiles of regular activities such as activities of daily living can support identifying trends in the data during critical events.A skeleton representation of the human body has been proven to be effective for this task.The skeletons are presented in graphs form-like.However,the topology of a graph is not structured like Euclideanbased data.Therefore,a new set of methods to perform the convolution operation upon the skeleton graph is proposed.Our proposal is based on the Spatial Temporal-Graph Convolutional Network(ST-GCN)framework.In this study,we proposed an improved set of label mapping methods for the ST-GCN framework.We introduce three split techniques(full distance split,connection split,and index split)as an alternative approach for the convolution operation.The experiments presented in this study have been trained using two benchmark datasets:NTU-RGB+D and Kinetics to evaluate the performance.Our results indicate that our split techniques outperform the previous partition strategies and aremore stable during training without using the edge importance weighting additional training parameter.Therefore,our proposal can provide a more realistic solution for real-time applications centred on daily living recognition systems activities for indoor environments.

A Graph with Adaptive AdjacencyMatrix for Relation Extraction

The relation is a semantic expression relevant to two named entities in a sentence.Since a sentence usually contains several named entities,it is essential to learn a structured sentence representation that encodes dependency information specific to the two named entities.In related work,graph convolutional neural networks are widely adopted to learn semantic dependencies,where a dependency tree initializes the adjacency matrix.However,this approach has two main issues.First,parsing a sentence heavily relies on external toolkits,which can be errorprone.Second,the dependency tree only encodes the syntactical structure of a sentence,which may not align with the relational semantic expression.In this paper,we propose an automatic graph learningmethod to autonomously learn a sentence’s structural information.Instead of using a fixed adjacency matrix initialized by a dependency tree,we introduce an Adaptive Adjacency Matrix to encode the semantic dependency between tokens.The elements of thismatrix are dynamically learned during the training process and optimized by task-relevant learning objectives,enabling the construction of task-relevant semantic dependencies within a sentence.Our model demonstrates superior performance on the TACRED and SemEval 2010 datasets,surpassing previous works by 1.3%and 0.8%,respectively.These experimental results show that our model excels in the relation extraction task,outperforming prior models.

Point Cloud Classification Network Based on Graph Convolution and Fusion Attention Mechanism

The classification of point cloud data is the key technology of point cloud data information acquisition and 3D reconstruction, which has a wide range of applications. However, the existing point cloud classification methods have some shortcomings when extracting point cloud features, such as insufficient extraction of local information and overlooking the information in other neighborhood features in the point cloud, and not focusing on the point cloud channel information and spatial information. To solve the above problems, a point cloud classification network based on graph convolution and fusion attention mechanism is proposed to achieve more accurate classification results. Firstly, the point cloud is regarded as a node on the graph, the k-nearest neighbor algorithm is used to compose the graph and the information between points is dynamically captured by stacking multiple graph convolution layers;then, with the assistance of 2D experience of attention mechanism, an attention mechanism which has the capability to integrate more attention to point cloud spatial and channel information is introduced to increase the feature information of point cloud, aggregate local useful features and suppress useless features. Through the classification experiments on ModelNet40 dataset, the experimental results show that compared with PointNet network without considering the local feature information of the point cloud, the average classification accuracy of the proposed model has a 4.4% improvement and the overall classification accuracy has a 4.4% improvement. Compared with other networks, the classification accuracy of the proposed model has also been improved.

Graph attention convolutional neural network model for chemical poisoning of honey bees’ prediction

The impact of pesticides on insect pollinators has caused worldwide concern. Both global bee decline and stopping the use of pesticides may have serious consequences for food security. Automated and accurate prediction of chemical poisoning of honey bees is a challenging task owing to a lack of understanding of chemical toxicity and introspection. Deep learning(DL) shows potential utility for general and highly variable tasks across fields. Here, we developed a new DL model of deep graph attention convolutional neural networks(GACNN) with the combination of undirected graph(UG) and attention convolutional neural networks(ACNN) to accurately classify chemical poisoning of honey bees. We used a training dataset of 720 pesticides and an external validation dataset of 90 pesticides, which is one order of magnitude larger than the previous datasets. We tested its performance in two ways: poisonous versus nonpoisonous and GACNN versus other frequently-used machine learning models. The first case represents the accuracy in identifying bee poisonous chemicals. The second represents performance advantages. The GACNN achieved ~6% higher performance for predicting toxic samples and more stable with ~7%Matthews Correlation Coefficient(MCC) higher compared to all tested models, demonstrating GACNN is capable of accurately classifying chemicals and has considerable potential in practical applications.In addition, we also summarized and evaluated the mechanisms underlying the response of honey bees to chemical exposure based on the mapping of molecular similarity. Moreover, our cloud platform(http://beetox.cn) of this model provides low-cost universal access to information, which could vitally enhance environmental risk assessment.

Stability and Generalization of Hypergraph Collaborative Networks

Graph neural networks have been shown to be very effective in utilizing pairwise relationships across samples.Recently,there have been several successful proposals to generalize graph neural networks to hypergraph neural networks to exploit more com-plex relationships.In particular,the hypergraph collaborative networks yield superior results compared to other hypergraph neural net-works for various semi-supervised learning tasks.The collaborative network can provide high quality vertex embeddings and hyperedge embeddings together by formulating them as a joint optimization problem and by using their consistency in reconstructing the given hy-pergraph.In this paper,we aim to establish the algorithmic stability of the core layer of the collaborative network and provide generaliz--ation guarantees.The analysis sheds light on the design of hypergraph filters in collaborative networks,for instance,how the data and hypergraph filters should be scaled to achieve uniform stability of the learning process.Some experimental results on real-world datasets are presented to illustrate the theory.

Aspect-Level Sentiment Analysis Based on Deep Learning

In recent years,deep learning methods have developed rapidly and found application in many fields,including natural language processing.In the field of aspect-level sentiment analysis,deep learning methods can also greatly improve the performance of models.However,previous studies did not take into account the relationship between user feature extraction and contextual terms.To address this issue,we use data feature extraction and deep learning combined to develop an aspect-level sentiment analysis method.To be specific,we design user comment feature extraction(UCFE)to distill salient features from users’historical comments and transform them into representative user feature vectors.Then,the aspect-sentence graph convolutional neural network(ASGCN)is used to incorporate innovative techniques for calculating adjacency matrices;meanwhile,ASGCN emphasizes capturing nuanced semantics within relationships among aspect words and syntactic dependency types.Afterward,three embedding methods are devised to embed the user feature vector into the ASGCN model.The empirical validations verify the effectiveness of these models,consistently surpassing conventional benchmarks and reaffirming the indispensable role of deep learning in advancing sentiment analysis methodologies.

Accelerating the Screening of Modified MA_(2)Z_(4) Catalysts for Hydrogen Evolution Reaction by Deep Learning-Based Local Geometric Analysis

Machine learning(ML)integrated with density functional theory(DFT)calculations have recently been used to accelerate the design and discovery of single-atom catalysts(SACs)by establishing deep structure–activity relationships.The traditional ML models are always difficult to identify the structural differences among the single-atom systems with different modification methods,leading to the limitation of the potential application range.Aiming to the structural properties of several typical two-dimensional MA_(2)Z_(4)-based single-atom systems(bare MA_(2)Z_(4) and metal single-atom doped/supported MA_(2)Z_(4)),an improved crystal graph convolutional neural network(CGCNN)classification model was employed,instead of the traditional machine learning regression model,to address the challenge of incompatibility in the studied systems.The CGCNN model was optimized using crystal graph representation in which the geometric configuration was divided into active layer,surface layer,and bulk layer(ASB-GCNN).Through ML and DFT calculations,five potential single-atom hydrogen evolution reaction(HER)catalysts were screened from chemical space of 600 MA_(2)Z_(4)-based materials,especially V_(1)/HfSn_(2)N_(4)(S)with high stability and activity(Δ_(GH*)is 0.06 eV).Further projected density of states(pDOS)analysis in combination with the wave function analysis of the SAC-H bond revealed that the SAC-dz^(2)orbital coincided with the H-s orbital around the energy level of−2.50 eV,and orbital analysis confirmed the formation ofσbonds.This study provides an efficient multistep screening design framework of metal single-atom catalyst for HER systems with similar two-dimensional supports but different geometric configurations.

GShuttle:Optimizing Memory Access Efficiency for Graph Convolu-tional Neural Network Accelerators

Graph convolutional neural networks(GCNs)have emerged as an effective approach to extending deep learning for graph data analytics,but they are computationally challenging given the irregular graphs and the large num-ber of nodes in a graph.GCNs involve chain sparse-dense matrix multiplications with six loops,which results in a large de-sign space for GCN accelerators.Prior work on GCN acceleration either employs limited loop optimization techniques,or determines the design variables based on random sampling,which can hardly exploit data reuse efficiently,thus degrading system efficiency.To overcome this limitation,this paper proposes GShuttle,a GCN acceleration scheme that maximizes memory access efficiency to achieve high performance and energy efficiency.GShuttle systematically explores loop opti-mization techniques for GCN acceleration,and quantitatively analyzes the design objectives(e.g.,required DRAM access-es and SRAM accesses)by analytical calculation based on multiple design variables.GShuttle further employs two ap-proaches,pruned search space sweeping and greedy search,to find the optimal design variables under certain design con-straints.We demonstrated the efficacy of GShuttle by evaluation on five widely used graph datasets.The experimental simulations show that GShuttle reduces the number of DRAM accesses by a factor of 1.5 and saves energy by a factor of 1.7 compared with the state-of-the-art approaches.

Aspect-Level Sentiment Analysis Incorporating Semantic and Syntactic Information

Aiming at the problem that existing models in aspect-level sentiment analysis cannot fully and effectively utilize sentence semantic and syntactic structure information, this paper proposes a graph neural network-based aspect-level sentiment classification model. Self-attention, aspectual word multi-head attention and dependent syntactic relations are fused and the node representations are enhanced with graph convolutional networks to enable the model to fully learn the global semantic and syntactic structural information of sentences. Experimental results show that the model performs well on three public benchmark datasets Rest14, Lap14, and Twitter, improving the accuracy of sentiment classification.

Weighted graph convolutional networks based on network node degree and efficiency

In the study of graph convolutional networks,the information aggregation of nodes is important for downstream tasks.However,current graph convolutional networks do not differentiate the importance of different neighboring nodes from the perspective of network topology when ag-gregating messages from neighboring nodes.Therefore,based on network topology,this paper proposes a weighted graph convolutional network based on network node degree and efficiency(W-GCN)model for semi-supervised node classification.To distinguish the importance of nodes,this paper uses the degree and the efficiency of nodes in the network to construct the impor-tance matrix of nodes,rather than the adjacency matrix,which usually is a normalized symmetry Laplacian matrix in graph convolutional network.So that weights of neighbor nodes can be as-signed respectively in the process of graph convolution operation.The proposed method is ex-amined through several real benchmark datasets(Cora,CiteSeer and PubMed)in the experimen-tal part.And compared with the graph convolutional network method.The experimental results show that the W-GCN model proposed in this paper is better than the graph convolutional net-work model in prediction accuracy and achieves better results.

Deep Learning Accelerates the Discovery of Two- Dimensional Catalysts for Hydrogen Evolution Reaction

Two-dimensional materials with active sites are expected to replace platinum as large-scale hydrogen production catalysts.However,the rapid discovery of excellent two-dimensional hydrogen evolution reaction catalysts is seriously hindered due to the long experiment cycle and the huge cost of high-throughput calculations of adsorption energies.Considering that the traditional regression models cannot consider all the potential sites on the surface of catalysts,we use a deep learning method with crystal graph convolutional neural networks to accelerate the discovery of high-performance two-dimensional hydrogen evolution reaction catalysts from two-dimensional materials database,with the prediction accuracy as high as 95.2%.The proposed method considers all active sites,screens out 38 high performance catalysts from 6,531 two-dimensional materials,predicts their adsorption energies at different active sites,and determines the potential strongest adsorption sites.The prediction accuracy of the two-dimensional hydrogen evolution reaction catalysts screening strategy proposed in this work is at the density-functional-theory level,but the prediction speed is 10.19 years ahead of the high-throughput screening,demonstrating the capability of crystal graph convolutional neural networks-deep learning method for efficiently discovering high-performance new structures over a wide catalytic materials space.

Judicious training pattern for superior molecular reorganization energy prediction model

Reorganization energy(RE)is closely related to the charge transport properties and is one of the important parameters for screening novel organic semiconductors(OSCs).With the rise of data-driven technology,accurate and efficient machine learning(ML)models for high-throughput screening novel organic molecules play an important role in the boom of material science.Comparing different molecular descriptors and algorithms,we construct a reasonable algorithm framework with molecular graphs to describe the compositional structure,convolutional neural networks to extract material features,and subsequently embedded fully connected neural networks to establish the mapping between features and predicted properties.With our well-designed judicious training pattern about feature-guided stratified random sampling,we have obtained a high-precision and robust reorganization energy prediction model,which can be used as one of the important descriptors for rapid screening potential OSCs.The root-meansquare error(RMSE)and the squared Pearson correlation coefficient(R^(2))of this model are 2.6 me V and0.99,respectively.More importantly,we confirm and emphasize that training pattern plays a crucial role in constructing supreme ML models.We are calling for more attention to designing innovative judicious training patterns in addition to high-quality databases,efficient material feature engineering and algorithm framework construction.

Railway Passenger Flow Forecasting by Integrating Passenger Flow Relationship and Spatiotemporal Similarity

Railway passenger flow forecasting can help to develop sensible railway schedules,make full use of railway resources,and meet the travel demand of passengers.The structure of passenger flow in railway networks and the spatiotemporal relationship of passenger flow among stations are two distinctive features of railway passenger flow.Most of the previous studies used only a single feature for prediction and lacked correlations,resulting in suboptimal performance.To address the above-mentioned problem,we proposed the railway passenger flow prediction model called Flow-Similarity Attention Graph Convolutional Network(F-SAGCN).First,we constructed the passenger flow relations graph(RG)based on the Origin-Destination(OD).Second,the Passenger Flow Fluctuation Similarity(PFFS)algorithm is used to measure the similarity of passenger flow between stations,which helps construct the spatiotemporal similarity graph(SG).Then,we determine the weights of the mutual influence of different stations at different times through an attention mechanism and extract spatiotemporal features through graph convolution on the RG and SG.Finally,we fused the spatiotemporal features and the original temporal features of stations for prediction.The comparison experiments on a railway bureau’s accurate railway passenger flow data show that the proposed F-SAGCN method improved the prediction accuracy and reduced the mean absolute percentage error(MAPE)of 46 stations to 7.93%.

A Flight Trajectory Prediction Method Based on Internal Relationships between Attributes

The rapid development of the aviation industry urgently requires airspace traffic management,and flight trajectory prediction is a core component of airspace traffic management.Flight trajectory is a multidimensional time series with rich spatio-temporal characteristics,and existing flight trajectory prediction methods only target the trajectory point temporal relationships,but not the implicit interrelationships among the trajectory point attributes.In this paper,a graph convolutional network(AR-GCN)based on the intra-attribute relationships is proposed for solving the flight track prediction problem.First,the network extracts the temporal features of each attribute and fuses them with the original features of the attribute to obtain the enhanced attribute features,then extracts the implicit relationships between attributes as inter-attribute relationship features.Secondly,the enhanced attribute features are used as nodes and the inter-attribute relationship features are used as edges to construct the inter-attribute relationship graph.Finally,the graph convolutional network is used to aggregate the attribute features.Based on the full fusion of the above features,we achieved high accuracy prediction of the trajectory.In this paper,experiments are conducted on ADS-B historical track data.We compare our method with the classical method and the proposed method.Experimental results show that our method achieves significant improvement in prediction accuracy.

Estimating posterior inference quality of the relational infinite latent feature model for overlapping community detection

Overlapping community detection has become a very hot research topic in recent decades,and a plethora of methods have been proposed.But,a common challenge in many existing overlapping community detection approaches is that the number of communities K must be predefined manually.We propose a flexible nonparametric Bayesian generative model for count-value networks,which can allow K to increase as more and more data are encountered instead of to be fixed in advance.The Indian buffet process was used to model the community assignment matrix Z,and an uncol-lapsed Gibbs sampler has been derived.However,as the community assignment matrix Zis a structured multi-variable parameter,how to summarize the posterior inference results andestimate the inference quality about Z,is still a considerable challenge in the literature.In this paper,a graph convolutional neural network based graph classifier was utilized to help tosummarize the results and to estimate the inference qualityabout Z.We conduct extensive experiments on synthetic data and real data,and find that empirically,the traditional posterior summarization strategy is reliable.

Maximizing the mechanical performance of Ti_(3)AlC_(2)-based MAX phases with aid of machine learning

Mechanical properties consisting of the bulk modulus,shear modulus,Young’s modulus,Poisson’s ratio,etc.,are key factors in determining the practical applications of MAX phases.These mechanical properties are mainly dependent on the strength of M–X and M–A bonds.In this study,a novel strategy based on the crystal graph convolution neural network(CGCNN)model has been successfully employed to tune these mechanical properties of Ti_(3)AlC_(2)-based MAX phases via the A-site substitution(Ti_(3)(Al1-xAx)C_(2)).The structure–property correlation between the A-site substitution and mechanical properties of Ti_(3)(Al1-xAx)C_(2)is established.The results show that the thermodynamic stability of Ti_(3)(Al1-xAx)C_(2)is enhanced with substitutions A=Ga,Si,Sn,Ge,Te,As,or Sb.The stiffness of Ti_(3)AlC_(2)increases with the substitution concentration of Si or As increasing,and the higher thermal shock resistance is closely associated with the substitution of Sn or Te.In addition,the plasticity of Ti_(3)AlC_(2)can be greatly improved when As,Sn,or Ge is used as a substitution.The findings and understandings demonstrated herein can provide universal guidance for the individual synthesis of high-performance MAX phases for various applications.