Continuous Sign Language Recognition Based on Spatial-Temporal Graph Attention Network

Continuous sign language recognition(CSLR)is challenging due to the complexity of video background,hand gesture variability,and temporal modeling difficulties.This work proposes a CSLR method based on a spatialtemporal graph attention network to focus on essential features of video series.The method considers local details of sign language movements by taking the information on joints and bones as inputs and constructing a spatialtemporal graph to reflect inter-frame relevance and physical connections between nodes.The graph-based multihead attention mechanism is utilized with adjacent matrix calculation for better local-feature exploration,and short-term motion correlation modeling is completed via a temporal convolutional network.We adopted BLSTM to learn the long-termdependence and connectionist temporal classification to align the word-level sequences.The proposed method achieves competitive results regarding word error rates(1.59%)on the Chinese Sign Language dataset and the mean Jaccard Index(65.78%)on the ChaLearn LAP Continuous Gesture Dataset.

A Novel Parameter-Optimized Recurrent Attention Network for Pipeline Leakage Detection

Accurate detection of pipeline leakage is essential to maintain the safety of pipeline transportation.Recently,deep learning(DL)has emerged as a promising tool for pipeline leakage detection(PLD).However,most existing DL methods have difficulty in achieving good performance in identifying leakage types due to the complex time dynamics of pipeline data.On the other hand,the initial parameter selection in the detection model is generally random,which may lead to unstable recognition performance.For this reason,a hybrid DL framework referred to as parameter-optimized recurrent attention network(PRAN)is presented in this paper to improve the accuracy of PLD.First,a parameter-optimized long short-term memory(LSTM)network is introduced to extract effective and robust features,which exploits a particle swarm optimization(PSO)algorithm with cross-entropy fitness function to search for globally optimal parameters.With this framework,the learning representation capability of the model is improved and the convergence rate is accelerated.Moreover,an anomaly-attention mechanism(AM)is proposed to discover class discriminative information by weighting the hidden states,which contributes to amplifying the normalabnormal distinguishable discrepancy,further improving the accuracy of PLD.After that,the proposed PRAN not only implements the adaptive optimization of network parameters,but also enlarges the contribution of normal-abnormal discrepancy,thereby overcoming the drawbacks of instability and poor generalization.Finally,the experimental results demonstrate the effectiveness and superiority of the proposed PRAN for PLD.

DHSEGATs:distance and hop-wise structures encoding enhanced graph attention networks

Numerous works prove that existing neighbor-averaging graph neural networks(GNNs)cannot efficiently catch structure features,and many works show that injecting structure,distance,position,or spatial features can significantly improve the performance of GNNs,however,injecting high-level structure and distance into GNNs is an intuitive but untouched idea.This work sheds light on this issue and proposes a scheme to enhance graph attention networks(GATs)by encoding distance and hop-wise structure statistics.Firstly,the hop-wise structure and distributional distance information are extracted based on several hop-wise ego-nets of every target node.Secondly,the derived structure information,distance information,and intrinsic features are encoded into the same vector space and then added together to get initial embedding vectors.Thirdly,the derived embedding vectors are fed into GATs,such as GAT and adaptive graph diffusion network(AGDN)to get the soft labels.Fourthly,the soft labels are fed into correct and smooth(C&S)to conduct label propagation and get final predictions.Experiments show that the distance and hop-wise structures encoding enhanced graph attention networks(DHSEGATs)achieve a competitive result.

Deep Attention Network for Pneumonia Detection Using Chest X-Ray Images

In computer vision,object recognition and image categorization have proven to be difficult challenges.They have,nevertheless,generated responses to a wide range of difficult issues from a variety of fields.Convolution Neural Networks(CNNs)have recently been identified as the most widely proposed deep learning(DL)algorithms in the literature.CNNs have unquestionably delivered cutting-edge achievements,particularly in the areas of image classification,speech recognition,and video processing.However,it has been noticed that the CNN-training assignment demands a large amount of data,which is in low supply,especially in the medical industry,and as a result,the training process takes longer.In this paper,we describe an attentionaware CNN architecture for classifying chest X-ray images to diagnose Pneumonia in order to address the aforementioned difficulties.AttentionModules provide attention-aware properties to the Attention Network.The attentionaware features of various modules alter as the layers become deeper.Using a bottom-up top-down feedforward structure,the feedforward and feedback attention processes are integrated into a single feedforward process inside each attention module.In the present work,a deep neural network(DNN)is combined with an attention mechanism to test the prediction of Pneumonia disease using chest X-ray pictures.To produce attention-aware features,the suggested networkwas built by merging channel and spatial attentionmodules in DNN architecture.With this network,we worked on a publicly available Kaggle chest X-ray dataset.Extensive testing was carried out to validate the suggested model.In the experimental results,we attained an accuracy of 95.47%and an F-score of 0.92,indicating that the suggested model outperformed against the baseline models.

Short‐term and long‐term memory self‐attention network for segmentation of tumours in 3D medical images

Tumour segmentation in medical images(especially 3D tumour segmentation)is highly challenging due to the possible similarity between tumours and adjacent tissues,occurrence of multiple tumours and variable tumour shapes and sizes.The popular deep learning‐based segmentation algorithms generally rely on the convolutional neural network(CNN)and Transformer.The former cannot extract the global image features effectively while the latter lacks the inductive bias and involves the complicated computation for 3D volume data.The existing hybrid CNN‐Transformer network can only provide the limited performance improvement or even poorer segmentation performance than the pure CNN.To address these issues,a short‐term and long‐term memory self‐attention network is proposed.Firstly,a distinctive self‐attention block uses the Transformer to explore the correlation among the region features at different levels extracted by the CNN.Then,the memory structure filters and combines the above information to exclude the similar regions and detect the multiple tumours.Finally,the multi‐layer reconstruction blocks will predict the tumour boundaries.Experimental results demonstrate that our method outperforms other methods in terms of subjective visual and quantitative evaluation.Compared with the most competitive method,the proposed method provides Dice(82.4%vs.76.6%)and Hausdorff distance 95%(HD95)(10.66 vs.11.54 mm)on the KiTS19 as well as Dice(80.2%vs.78.4%)and HD95(9.632 vs.12.17 mm)on the LiTS.

Topic-Aware Abstractive Summarization Based on Heterogeneous Graph Attention Networks for Chinese Complaint Reports

Automatic text summarization(ATS)plays a significant role in Natural Language Processing(NLP).Abstractive summarization produces summaries by identifying and compressing the most important information in a document.However,there are only relatively several comprehensively evaluated abstractive summarization models that work well for specific types of reports due to their unstructured and oral language text characteristics.In particular,Chinese complaint reports,generated by urban complainers and collected by government employees,describe existing resident problems in daily life.Meanwhile,the reflected problems are required to respond speedily.Therefore,automatic summarization tasks for these reports have been developed.However,similar to traditional summarization models,the generated summaries still exist problems of informativeness and conciseness.To address these issues and generate suitably informative and less redundant summaries,a topic-based abstractive summarization method is proposed to obtain global and local features.Additionally,a heterogeneous graph of the original document is constructed using word-level and topic-level features.Experiments and analyses on public review datasets(Yelp and Amazon)and our constructed dataset(Chinese complaint reports)show that the proposed framework effectively improves the performance of the abstractive summarization model for Chinese complaint reports.

Semantic Path Attention Network Based on Heterogeneous Graphs for Natural Language to SQL Task

The natural language to SQL(NL2SQL)task is an emerging research area that aims to transform a natural language with a given database into an SQL query.The earlier approaches were to process the input into a heterogeneous graph.However,previous models failed to distinguish the types of multi-hop connections of the heterogeneous graph,which tended to ignore crucial semantic path information.To this end,a two-layer attention network is presented to focus on essential neighbor nodes and mine enlightening semantic paths for feature encoding.The weighted edge is introduced for schema linking to connect the nodes with semantic similarity.In the decoding phase,a rule-based pruning strategy is offered to refine the generated SQL queries.From the experimental results,the approach is shown to learn a good encoding representation and decode the representation to generate results with practical meaning.

Learning a Discriminative Feature Attention Network for pancreas CT segmentation

Accurate pancreas segmentation is critical for the diagnosis and management of diseases of the pancreas. It is challenging to precisely delineate pancreas due to the highly variations in volume, shape and location. In recent years, coarse-to-fine methods have been widely used to alleviate class imbalance issue and improve pancreas segmentation accuracy. However,cascaded methods could be computationally intensive and the refined results are significantly dependent on the performance of its coarse segmentation results. To balance the segmentation accuracy and computational efficiency, we propose a Discriminative Feature Attention Network for pancreas segmentation, to effectively highlight pancreas features and improve segmentation accuracy without explicit pancreas location. The final segmentation is obtained by applying a simple yet effective post-processing step. Two experiments on both public NIH pancreas CT dataset and abdominal BTCV multi-organ dataset are individually conducted to show the effectiveness of our method for 2 D pancreas segmentation. We obtained average Dice Similarity Coefficient(DSC) of 82.82±6.09%, average Jaccard Index(JI) of 71.13± 8.30% and average Symmetric Average Surface Distance(ASD) of 1.69 ± 0.83 mm on the NIH dataset. Compared to the existing deep learning-based pancreas segmentation methods, our experimental results achieve the best average DSC and JI value.

Stacked Attention Networks for Referring Expressions Comprehension

Referring expressions comprehension is the task of locating the image region described by a natural language expression,which refer to the properties of the region or the relationships with other regions.Most previous work handles this problem by selecting the most relevant regions from a set of candidate regions,when there are many candidate regions in the set these methods are inefficient.Inspired by recent success of image captioning by using deep learning methods,in this paper we proposed a framework to understand the referring expressions by multiple steps of reasoning.We present a model for referring expressions comprehension by selecting the most relevant region directly from the image.The core of our model is a recurrent attention network which can be seen as an extension of Memory Network.The proposed model capable of improving the results by multiple computational hops.We evaluate the proposed model on two referring expression datasets:Visual Genome and Flickr30k Entities.The experimental results demonstrate that the proposed model outperform previous state-of-the-art methods both in accuracy and efficiency.We also conduct an ablation experiment to show that the performance of the model is not getting better with the increase of the attention layers.

Air Pollution Prediction Via Graph Attention Network and Gated Recurrent Unit

PM2.5 concentration prediction is of great significance to environmental protection and human health.Achieving accurate prediction of PM2.5 concentration has become an important research task.However,PM2.5 pollutants can spread in the earth’s atmosphere,causing mutual influence between different cities.To effectively capture the air pollution relationship between cities,this paper proposes a novel spatiotemporal model combining graph attention neural network(GAT)and gated recurrent unit(GRU),named GAT-GRU for PM2.5 concentration prediction.Specifically,GAT is used to learn the spatial dependence of PM2.5 concentration data in different cities,and GRU is to extract the temporal dependence of the long-term data series.The proposed model integrates the learned spatio-temporal dependencies to capture long-term complex spatio-temporal features.Considering that air pollution is related to the meteorological conditions of the city,the knowledge acquired from meteorological data is used in the model to enhance PM2.5 prediction performance.The input of the GAT-GRU model consists of PM2.5 concentration data and meteorological data.In order to verify the effectiveness of the proposed GAT-GRU prediction model,this paper designs experiments on real-world datasets compared with other baselines.Experimental results prove that our model achieves excellent performance in PM2.5 concentration prediction.

Label-Aware Chinese Event Detection with Heterogeneous Graph Attention Network

Event detection(ED)seeks to recognize event triggers and classify them into the predefined event types.Chinese ED is formulated as a character-level task owing to the uncertain word boundaries.Prior methods try to incorpo-rate word-level information into characters to enhance their semantics.However,they experience two problems.First,they fail to incorporate word-level information into each character the word encompasses,causing the insufficient word-charac-ter interaction problem.Second,they struggle to distinguish events of similar types with limited annotated instances,which is called the event confusing problem.This paper proposes a novel model named Label-Aware Heterogeneous Graph Attention Network(L-HGAT)to address these two problems.Specifically,we first build a heterogeneous graph of two node types and three edge types to maximally preserve word-character interactions,and then deploy a heterogeneous graph attention network to enhance the semantic propagation between characters and words.Furthermore,we design a pushing-away game to enlarge the predicting gap between the ground-truth event type and its confusing counterpart for each character.Experimental results show that our L-HGAT model consistently achieves superior performance over prior competitive methods.

The Short-Term Prediction ofWind Power Based on the Convolutional Graph Attention Deep Neural Network

The fluctuation of wind power affects the operating safety and power consumption of the electric power grid and restricts the grid connection of wind power on a large scale.Therefore,wind power forecasting plays a key role in improving the safety and economic benefits of the power grid.This paper proposes a wind power predicting method based on a convolutional graph attention deep neural network with multi-wind farm data.Based on the graph attention network and attention mechanism,the method extracts spatial-temporal characteristics from the data of multiple wind farms.Then,combined with a deep neural network,a convolutional graph attention deep neural network model is constructed.Finally,the model is trained with the quantile regression loss function to achieve the wind power deterministic and probabilistic prediction based on multi-wind farm spatial-temporal data.A wind power dataset in the U.S.is taken as an example to demonstrate the efficacy of the proposed model.Compared with the selected baseline methods,the proposed model achieves the best prediction performance.The point prediction errors(i.e.,root mean square error(RMSE)and normalized mean absolute percentage error(NMAPE))are 0.304 MW and 1.177%,respectively.And the comprehensive performance of probabilistic prediction(i.e.,con-tinuously ranked probability score(CRPS))is 0.580.Thus,the significance of multi-wind farm data and spatial-temporal feature extraction module is self-evident.

MSADCN:Multi-Scale Attentional Densely Connected Network for Automated Bone Age Assessment

Bone age assessment(BAA)helps doctors determine how a child’s bones grow and develop in clinical medicine.Traditional BAA methods rely on clinician expertise,leading to time-consuming predictions and inaccurate results.Most deep learning-based BAA methods feed the extracted critical points of images into the network by providing additional annotations.This operation is costly and subjective.To address these problems,we propose a multi-scale attentional densely connected network(MSADCN)in this paper.MSADCN constructs a multi-scale dense connectivity mechanism,which can avoid overfitting,obtain the local features effectively and prevent gradient vanishing even in limited training data.First,MSADCN designs multi-scale structures in the densely connected network to extract fine-grained features at different scales.Then,coordinate attention is embedded to focus on critical features and automatically locate the regions of interest(ROI)without additional annotation.In addition,to improve the model’s generalization,transfer learning is applied to train the proposed MSADCN on the public dataset IMDB-WIKI,and the obtained pre-trained weights are loaded onto the Radiological Society of North America(RSNA)dataset.Finally,label distribution learning(LDL)and expectation regression techniques are introduced into our model to exploit the correlation between hand bone images of different ages,which can obtain stable age estimates.Extensive experiments confirm that our model can converge more efficiently and obtain a mean absolute error(MAE)of 4.64 months,outperforming some state-of-the-art BAA methods.

Graph attention network for global search of atomic clusters:A case study of Ag_(n)(n=14-26)clusters

Due to coexistence of huge number of structural isomers,global search for the ground-state structures of atomic clusters is a challenging issue.The difficulty also originates from the computational cost of ab initio methods for describing the potential energy surface.Recently,machine learning techniques have been widely utilized to accelerate materials discovery and molecular simulation.Compared to the commonly used artificial neural network,graph network is naturally suitable for clusters with flexible geometric environment of each atom.Herein we develop a cluster graph attention network(CGANet)by aggregating information of neighboring vertices and edges using attention mechanism,which can precisely predict the binding energy and force of silver clusters with root mean square error of 5.4 meV/atom and mean absolute error of 42.3 meV/Å,respectively.As a proof-of-concept,we have performed global optimization of mediumsized Agn clusters(n=14–26)by combining CGANet and genetic algorithm.The reported ground-state structures for n=14–21,have been successfully reproduced,while entirely new lowest-energy structures are obtained for n=22–26.In addition to the description of potential energy surface,the CGANet is also applied to predict the electronic properties of clusters,such as HOMO energy and HOMO-LUMO gap.With accuracy comparable to ab initio methods and acceleration by at least two orders of magnitude,CGANet holds great promise in global search of lowest-energy structures of large clusters and inverse design of functional clusters.

Integrating multi-modal information to detect spatial domains of spatial transcriptomics by graph attention network

Recent advances in spatially resolved transcriptomic technologies have enabled unprecedented opportunities to elucidate tissue architecture and function in situ.Spatial transcriptomics can provide multimodal and complementary information simultaneously,including gene expression profiles,spatial locations,and histology images.However,most existing methods have limitations in efficiently utilizing spatial information and matched high-resolution histology images.To fully leverage the multi-modal information,we propose a SPAtially embedded Deep Attentional graph Clustering(SpaDAC)method to identify spatial domains while reconstructing denoised gene expression profiles.This method can efficiently learn the low-dimensional embeddings for spatial transcriptomics data by constructing multi-view graph modules to capture both spatial location connectives and morphological connectives.Benchmark results demonstrate that SpaDAC outperforms other algorithms on several recent spatial transcriptomics datasets.SpaDAC is a valuable tool for spatial domain detection,facilitating the comprehension of tissue architecture and cellular microenvironment.The source code of SpaDAC is freely available at Github(https://github.com/huoyuying/SpaDAC.git).

Crysformer:An attention-based graph neural network for properties prediction of crystals

We present a novel approach for the prediction of crystal material properties that is distinct from the computationally complex and expensive density functional theory(DFT)-based calculations.Instead,we utilize an attention-based graph neural network that yields high-accuracy predictions.Our approach employs two attention mechanisms that allow for message passing on the crystal graphs,which in turn enable the model to selectively attend to pertinent atoms and their local environments,thereby improving performance.We conduct comprehensive experiments to validate our approach,which demonstrates that our method surpasses existing methods in terms of predictive accuracy.Our results suggest that deep learning,particularly attention-based networks,holds significant promise for predicting crystal material properties,with implications for material discovery and the refined intelligent systems.

Hard-rock tunnel lithology identification using multiscale dilated convolutional attention network based on tunnel face images

For real-time classification of rock-masses in hard-rock tunnels,quick determination of the rock lithology on the tunnel face during construction is essential.Motivated by current breakthroughs in artificial intelligence technology in machine vision,a new automatic detection approach for classifying tunnel lithology based on tunnel face images was developed.The method benefits from residual learning for training a deep convolutional neural network(DCNN),and a multi-scale dilated convolutional attention block is proposed.The block with different dilation rates can provide various receptive fields,and thus it can extract multi-scale features.Moreover,the attention mechanism is utilized to select the salient features adaptively and further improve the performance of the model.In this study,an initial image data set made up of photographs of tunnel faces consisting of basalt,granite,siltstone,and tuff was first collected.After classifying and enhancing the training,validation,and testing data sets,a new image data set was generated.A comparison of the experimental findings demonstrated that the suggested approach outperforms previous classifiers in terms of various indicators,including accuracy,precision,recall,F1-score,and computing time.Finally,a visualization analysis was performed to explain the process of the network in the classification of tunnel lithology through feature extraction.Overall,this study demonstrates the potential of using artificial intelligence methods for in situ rock lithology classification utilizing geological images of the tunnel face.

Speech Emotion Recognition Using Cascaded Attention Network with Joint Loss for Discrimination of Confusions

Due to the complexity of emotional expression, recognizing emotions from the speech is a critical and challenging task. In most of the studies, some specific emotions are easily classified incorrectly. In this paper, we propose a new framework that integrates cascade attention mechanism and joint loss for speech emotion recognition (SER), aiming to solve feature confusions for emotions that are difficult to be classified correctly. First, we extract the mel frequency cepstrum coefficients (MFCCs), deltas, and delta-deltas from MFCCs to form 3-dimensional (3D) features, thus effectively reducing the interference of external factors. Second, we employ spatiotemporal attention to selectively discover target emotion regions from the input features, where self-attention with head fusion captures the long-range dependency of temporal features. Finally, the joint loss function is employed to distinguish emotional embeddings with high similarity to enhance the overall performance. Experiments on interactive emotional dyadic motion capture (IEMOCAP) database indicate that the method achieves a positive improvement of 2.49% and 1.13% in weighted accuracy (WA) and unweighted accuracy (UA), respectively, compared to the state-of-the-art strategies.

Improving Wind Forecasts Using a Gale-Aware Deep Attention Network

Numerical weather prediction of wind speed requires statistical postprocessing of systematic errors to obtain reliable and accurate forecasts.However,use of postprocessing models is often undesirable for extreme weather events such as gales.Here,we propose a postprocessing algorithm based on a gale-aware deep attention network to simultaneously improve wind speed forecasts and gale area warnings.Specifically,the algorithm includes both a galeaware loss function that focuses the model on potential gale areas,and an observation station supervision strategy that alleviates the problem of missing extreme values caused by data gridding.The effectiveness of the proposed model was verified by using data from 235 wind speed observation stations.Experimental results show that our model can produce wind speed forecasts with a root-mean-square error of 1.1547 m s^(-1),and a Hanssen–Kuipers discriminant score of 0.517,performance that is superior to that of the other postprocessing algorithms considered.

Social Robot Detection Method with Improved Graph Neural Networks

Social robot accounts controlled by artificial intelligence or humans are active in social networks,bringing negative impacts to network security and social life.Existing social robot detection methods based on graph neural networks suffer from the problem of many social network nodes and complex relationships,which makes it difficult to accurately describe the difference between the topological relations of nodes,resulting in low detection accuracy of social robots.This paper proposes a social robot detection method with the use of an improved neural network.First,social relationship subgraphs are constructed by leveraging the user’s social network to disentangle intricate social relationships effectively.Then,a linear modulated graph attention residual network model is devised to extract the node and network topology features of the social relation subgraph,thereby generating comprehensive social relation subgraph features,and the feature-wise linear modulation module of the model can better learn the differences between the nodes.Next,user text content and behavioral gene sequences are extracted to construct social behavioral features combined with the social relationship subgraph features.Finally,social robots can be more accurately identified by combining user behavioral and relationship features.By carrying out experimental studies based on the publicly available datasets TwiBot-20 and Cresci-15,the suggested method’s detection accuracies can achieve 86.73%and 97.86%,respectively.Compared with the existing mainstream approaches,the accuracy of the proposed method is 2.2%and 1.35%higher on the two datasets.The results show that the method proposed in this paper can effectively detect social robots and maintain a healthy ecological environment of social networks.