Multivariate Time Series Anomaly Detection Based on Spatial-Temporal Network and Transformer in Industrial Internet of Things

In the Industrial Internet of Things(IIoT),sensors generate time series data to reflect the working state.When the systems are attacked,timely identification of outliers in time series is critical to ensure security.Although many anomaly detection methods have been proposed,the temporal correlation of the time series over the same sensor and the state(spatial)correlation between different sensors are rarely considered simultaneously in these methods.Owing to the superior capability of Transformer in learning time series features.This paper proposes a time series anomaly detection method based on a spatial-temporal network and an improved Transformer.Additionally,the methods based on graph neural networks typically include a graph structure learning module and an anomaly detection module,which are interdependent.However,in the initial phase of training,since neither of the modules has reached an optimal state,their performance may influence each other.This scenario makes the end-to-end training approach hard to effectively direct the learning trajectory of each module.This interdependence between the modules,coupled with the initial instability,may cause the model to find it hard to find the optimal solution during the training process,resulting in unsatisfactory results.We introduce an adaptive graph structure learning method to obtain the optimal model parameters and graph structure.Experiments on two publicly available datasets demonstrate that the proposed method attains higher anomaly detection results than other methods.

A Spatial-Temporal Network Perspective for the Propagation Dynamics of Air Traffic Delays

Intractable delays occur in air traffic due to the imbalance between ever-increasing air traffic demand and limited airspace capacity.As air traffic is associated with complex air transport systems,delays can be magnified and propagated throughout these systems,resulting in the emergent behavior known as delay propagation.An understanding of delay propagation dynamics is pertinent to modern air traffic management.In this work,we present a complex network perspective of delay propagation dynamics.Specifically,we model air traffic scenarios using spatial–temporal networks with airports as the nodes.To establish the dynamic edges between the nodes,we develop a delay propagation method and apply it to a given set of air traffic schedules.Based on the constructed spatial-temporal networks,we suggest three metrics-magnitude,severity,and speed-to gauge delay propagation dynamics.To validate the effectiveness of the proposed method,we carry out case studies on domestic flights in the Southeastern Asia region(SAR)and the United States.Experiments demonstrate that the propagation magnitude in terms of the number of flights affected by delay propagation and the amount of propagated delays for the US traffic are respectively five and ten times those of the SAR.Experiments further reveal that the propagation speed for US traffic is eight times faster than that of the SAR.The delay propagation dynamics reveal that about six hub airports in the SAR have significant propagated delays,while the situation in the United States is considerably worse,with a corresponding number of around 16.This work provides a potent tool for tracing the evolution of air traffic delays.

AFSTGCN:Prediction for multivariate time series using an adaptive fused spatial-temporal graph convolutional network

The prediction for Multivariate Time Series(MTS)explores the interrelationships among variables at historical moments,extracts their relevant characteristics,and is widely used in finance,weather,complex industries and other fields.Furthermore,it is important to construct a digital twin system.However,existing methods do not take full advantage of the potential properties of variables,which results in poor predicted accuracy.In this paper,we propose the Adaptive Fused Spatial-Temporal Graph Convolutional Network(AFSTGCN).First,to address the problem of the unknown spatial-temporal structure,we construct the Adaptive Fused Spatial-Temporal Graph(AFSTG)layer.Specifically,we fuse the spatial-temporal graph based on the interrelationship of spatial graphs.Simultaneously,we construct the adaptive adjacency matrix of the spatial-temporal graph using node embedding methods.Subsequently,to overcome the insufficient extraction of disordered correlation features,we construct the Adaptive Fused Spatial-Temporal Graph Convolutional(AFSTGC)module.The module forces the reordering of disordered temporal,spatial and spatial-temporal dependencies into rule-like data.AFSTGCN dynamically and synchronously acquires potential temporal,spatial and spatial-temporal correlations,thereby fully extracting rich hierarchical feature information to enhance the predicted accuracy.Experiments on different types of MTS datasets demonstrate that the model achieves state-of-the-art single-step and multi-step performance compared with eight other deep learning models.

Adaptive spatial-temporal graph attention network for traffic speed prediction

Considering the nonlinear structure and spatial-temporal correlation of traffic network,and the influence of potential correlation between nodes of traffic network on the spatial features,this paper proposes a traffic speed prediction model based on the combination of graph attention network with self-adaptive adjacency matrix(SAdpGAT)and bidirectional gated recurrent unit(BiGRU).First-ly,the model introduces graph attention network(GAT)to extract the spatial features of real road network and potential road network respectively in spatial dimension.Secondly,the spatial features are input into BiGRU to extract the time series features.Finally,the prediction results of the real road network and the potential road network are connected to generate the final prediction results of the model.The experimental results show that the prediction accuracy of the proposed model is im-proved obviously on METR-LA and PEMS-BAY datasets,which proves the advantages of the pro-posed spatial-temporal model in traffic speed prediction.

Predicting Traffic Flow Using Dynamic Spatial-Temporal Graph Convolution Networks

Traffic flow prediction plays a key role in the construction of intelligent transportation system.However,due to its complex spatio-temporal dependence and its uncertainty,the research becomes very challenging.Most of the existing studies are based on graph neural networks that model traffic flow graphs and try to use fixed graph structure to deal with the relationship between nodes.However,due to the time-varying spatial correlation of the traffic network,there is no fixed node relationship,and these methods cannot effectively integrate the temporal and spatial features.This paper proposes a novel temporal-spatial dynamic graph convolutional network(TSADGCN).The dynamic time warping algorithm(DTW)is introduced to calculate the similarity of traffic flow sequence among network nodes in the time dimension,and the spatiotemporal graph of traffic flow is constructed to capture the spatiotemporal characteristics and dependencies of traffic flow.By combining graph attention network and time attention network,a spatiotemporal convolution block is constructed to capture spatiotemporal characteristics of traffic data.Experiments on open data sets PEMSD4 and PEMSD8 show that TSADGCN has higher prediction accuracy than well-known traffic flow prediction algorithms.

Multi-Head Attention Spatial-Temporal Graph Neural Networks for Traffic Forecasting

Accurate traffic prediction is crucial for an intelligent traffic system (ITS). However, the excessive non-linearity and complexity of the spatial-temporal correlation in traffic flow severely limit the prediction accuracy of most existing models, which simply stack temporal and spatial modules and fail to capture spatial-temporal features effectively. To improve the prediction accuracy, a multi-head attention spatial-temporal graph neural network (MSTNet) is proposed in this paper. First, the traffic data is decomposed into unique time spans that conform to positive rules, and valuable traffic node attributes are mined through an adaptive graph structure. Second, time and spatial features are captured using a multi-head attention spatial-temporal module. Finally, a multi-step prediction module is used to achieve future traffic condition prediction. Numerical experiments were conducted on an open-source dataset, and the results demonstrate that MSTNet performs well in spatial-temporal feature extraction and achieves more positive forecasting results than the baseline methods.

Combined spatial-temporal energy harvesting and relay selection for cognitive wireless powered networks

In order to improve the Energy Efficiency(EE)and spectrum utilization of Cognitive Wireless Powered Networks(CWPNs),a combined spatial-temporal Energy Harvesting(EH)and relay selection scheme is proposed.In the proposed scheme,for protecting the Primary User(PU),a two-layer guard zone is set outside the PU based on the outage probability threshold of the PU.Moreover,to increase the energy of the CWPNs,the EH zone in the two-layer guard zone allows the Secondary Users(SUs)to spatially harvest energy from the Radio Frequency(RF)signals of temporally active PUs.To improve the utilization of the PU spectrum,the guard zone outside the EH zone allows for the constrained power transmission of SUs.Moreover,the relay selection transmission is designed in the transmission zone of the SU to improve the EE of the CWPNs.In addition to the EE of the CWPNs,the outage probabilities of the SU and PU are derived.The results reveal that the setting of a two-layer guard zone can effectively reduce the outage probability of the PU and improve the EE of CWPNs.Furthermore,the relay selection transmission decreases the outage probabilities of the SUs.

Comprehensive evaluation and spatial-temporal evolution characteristics of urban resilience in Chengdu-Chongqing Economic Circle

To clarify the connotations and extensions of urban resilience,this study focuses on the Chengdu-Chongqing Economic Circle with 16 cities as research subjects.A comprehensive evaluation index system was constructed to measure the resilience of each city from 2003 to 2020.The spatial-temporal evolution characteristics were analyzed using Kernel density estimation,standard deviation ellipse,and spatial Markov chain analysis,and the spatial Tobit model was introduced to discover the influencing factors.The results indicate the following:①Urban resilience in the Chengdu-Chongqing Economic Circle displays an upward trend,with the center of gravity moving to the southwest,and the polarization phenomenon intensifying.②The urban resilience level in a region has certain spatial and geographical dependence,while the probability of urban resilience transfer differs in adjacent cities with different resilience levels.③Urban centrality,economic scale,openness level,and financial development promote urban resilience,whereas government scale significantly inhibits it.Finally,this paper proposes countermeasures and suggestions to improve the urban resilience of the Chengdu-Chongqing Economic Circle.

ST-Trader:A Spatial-Temporal Deep Neural Network for Modeling Stock Market Movement

Stocks that are fundamentally connected with each other tend to move together.Considering such common trends is believed to benefit stock movement forecasting tasks.However,such signals are not trivial to model because the connections among stocks are not physically presented and need to be estimated from volatile data.Motivated by this observation,we propose a framework that incorporates the inter-connection of firms to forecast stock prices.To effectively utilize a large set of fundamental features,we further design a novel pipeline.First,we use variational autoencoder(VAE)to reduce the dimension of stock fundamental information and then cluster stocks into a graph structure(fundamentally clustering).Second,a hybrid model of graph convolutional network and long-short term memory network(GCN-LSTM)with an adjacency graph matrix(learnt from VAE)is proposed for graph-structured stock market forecasting.Experiments on minute-level U.S.stock market data demonstrate that our model effectively captures both spatial and temporal signals and achieves superior improvement over baseline methods.The proposed model is promising for other applications in which there is a possible but hidden spatial dependency to improve time-series prediction.

Spatial-temporal distribution and geochemistry of highly evolved Mesozoic granites in Great Xing’an Range,NE China:Discriminant criteria and geological significance

Highly evolved granite is an important sign of the mature continent crust and closely associated with deposits of rare metals.In this work,the authors undertake systematically zircon U-Pb ages and whole rock elemental data for highly evolved granitic intrusions from the Great Xing’an Range(GXR),NE China,to elucidate their discriminant criteria,spatial-temporal distribution,differentiation and geodynamic mecha-nism.Geochemical data of these highly evolved granites suggest that high w(SiO_(2))(>70%)and differentiation index(DI>88)could be quantified indicators,while strong Eu depletion,high TE_(1,3),lowΣREE and low Zr/Hf,Nb/Ta,K/Rb could only be qualitative indicators.Zircon U-Pb ages suggest that the highly evolved gran-ites in the GXR were mainly formed in Late Mesozoic,which can be divided into two major stages:Late Ju-rassic-early Early Cretaceous(162-136 Ma,peak at 138 Ma),and late Early Cretaceous(136-106 Ma,peak at 126 Ma).The highly evolved granites are mainly distributed in the central-southern GXR,and display a weakly trend of getting younger from northwest to southeast,meanwhile indicating the metallogenic potential of rare metals within the central GXR.The spatial-temporal distribution,combined with regional geological data,indicates the highly evolved Mesozoic granites in the GXR were emplaced in an extensional environ-ment,of which the Late Jurassic-early Early Cretaceous extension was related to the closure of the Mongol-Okhotsk Ocean and roll-back of the Paleo-Pacific Plate,while the late Early Cretaceous extension was mainly related to the roll-back of the Paleo-Pacific Plate.

Spatial-temporal Divergence Characteristics and Driving Factors of Green Economic Efficiency in the Yangtze River Economic Belt of China

The spatial and temporal variation of green economic efficiency and its driving factors are of great significance for the construction of high-efficiency and low-consumption green development model and sustainable socio-economic development.The research focused on the Yangtze River Economic Belt(YREB)and employed the miniumum distance to strong efficient frontier DEA(MinDs)model to measure the green economic efficiency of the municipalities in the region between 2008 and 2020.Then,the spatial autocorrelation model was used to analyze the evolution characteristics of its spatial pattern.Finally,Geodetector was applied to reveal the drivers and their interactions on green economic efficiency.It is found that:1)the overall green economic efficiency of the YREB from 2008 to 2020 shows a W-shaped fluctuating upward trend,green economic efficiency is greater in the downstream and smallest in the upstream;2)the spatial distribution of green economic efficiency shows clustering characteristics,with multi-core clustering based on‘city clusters-central cities'becoming more obvious over time;the High-High agglomeration type is mainly clustered in Jiangsu and Zheji-ang,while the Low-Low agglomeration type is clustered in the western Sichuan Plateau area and southwestern Yunnan;3)from input-output factors,whether it is the YREB as a whole or the upper,middle and lower reaches regions,the economic development level,labor input,and capital investment are the leading factors in the spatial-temporal evolution of green economic efficiency,with the com-prehensive influence of economic development level and pollution index being the most important interactive driving factor;4)from so-cio-economic factors,information technology drivers such as government intervention,transportation accessibility,information infra-structure,and Internet penetration are always high impact influencers and dominant interaction factors for green economic efficiency in the YREB and the three major regions in the upper,middle and lower reaches.Accordingly,the article puts forward relevant policy re-commendations in terms of formulating differentiated green transformation strategies,strengthening network leadership and informa-tion technology construction and coordinating multi-factor integrated development,which could provide useful reference for promoting synergistic green economic efficiency in the YREB.

Dense Spatial-Temporal Graph Convolutional Network Based on Lightweight OpenPose for Detecting Falls

Fall behavior is closely related to high mortality in the elderly,so fall detection becomes an important and urgent research area.However,the existing fall detection methods are difficult to be applied in daily life due to a large amount of calculation and poor detection accuracy.To solve the above problems,this paper proposes a dense spatial-temporal graph convolutional network based on lightweight OpenPose.Lightweight OpenPose uses MobileNet as a feature extraction network,and the prediction layer uses bottleneck-asymmetric structure,thus reducing the amount of the network.The bottleneck-asymmetrical structure compresses the number of input channels of feature maps by 1×1 convolution and replaces the 7×7 convolution structure with the asymmetric structure of 1×7 convolution,7×1 convolution,and 7×7 convolution in parallel.The spatial-temporal graph convolutional network divides the multi-layer convolution into dense blocks,and the convolutional layers in each dense block are connected,thus improving the feature transitivity,enhancing the network’s ability to extract features,thus improving the detection accuracy.Two representative datasets,Multiple Cameras Fall dataset(MCF),and Nanyang Technological University Red Green Blue+Depth Action Recognition dataset(NTU RGB+D),are selected for our experiments,among which NTU RGB+D has two evaluation benchmarks.The results show that the proposed model is superior to the current fall detection models.The accuracy of this network on the MCF dataset is 96.3%,and the accuracies on the two evaluation benchmarks of the NTU RGB+D dataset are 85.6%and 93.5%,respectively.

Spatial-temporal Variation Characteristics of Water Quality in the Lower Reaches of the Nenjiang River

As an important river in the western part of Jilin Province,the lower reach of the Nenjiang River is an important wetland water source conservation area in Jilin Province.Within the watershed,it governs the Momoge Wetland,the Xianghai Wetland,and the Danjiang Wetland in Jilin Province.The main problem in the lower reaches of the Nenjiang River is the uneven distribution of water resources in time and space,and the intensification of land salinization.Zhenlai County and Da an City in the Nenjiang River Basin have sufficient surface water resources,with surface water as the drinking water source.Baicheng City and Tongyu County have scarce surface water resources,and both use groundwater as their domestic water source.The main polluted section in the basin is the Xianghai Reservoir,and the annual water quality evaluation is Class V.However,the water quality of the Tao er River,the main stream of the Nenjiang River,is significantly better than that of the Xianghai Reservoir.In order to better study the water environmental pollution situation in the Nenjiang River basin,monitoring data from five sections of non seasonal rivers in the basin from 2012 to 2021 were selected for studying water quality.This in-depth exploration of the water pollution status and river water quality change trends in the Nenjiang River basin is of great significance for future rural development,agricultural pattern transformation,and the promotion of water ecological civilization construction.

Continuous Sign Language Recognition Based on Spatial-Temporal Graph Attention Network

Continuous sign language recognition(CSLR)is challenging due to the complexity of video background,hand gesture variability,and temporal modeling difficulties.This work proposes a CSLR method based on a spatialtemporal graph attention network to focus on essential features of video series.The method considers local details of sign language movements by taking the information on joints and bones as inputs and constructing a spatialtemporal graph to reflect inter-frame relevance and physical connections between nodes.The graph-based multihead attention mechanism is utilized with adjacent matrix calculation for better local-feature exploration,and short-term motion correlation modeling is completed via a temporal convolutional network.We adopted BLSTM to learn the long-termdependence and connectionist temporal classification to align the word-level sequences.The proposed method achieves competitive results regarding word error rates(1.59%)on the Chinese Sign Language dataset and the mean Jaccard Index(65.78%)on the ChaLearn LAP Continuous Gesture Dataset.

Application of virtual reality technology improves the functionality of brain networks in individuals experiencing pain

Medical procedures are inherently invasive and carry the risk of inducing pain to the mind and body.Recently,efforts have been made to alleviate the discomfort associated with invasive medical procedures through the use of virtual reality(VR)technology.VR has been demonstrated to be an effective treatment for pain associated with medical procedures,as well as for chronic pain conditions for which no effective treatment has been established.The precise mechanism by which the diversion from reality facilitated by VR contributes to the diminution of pain and anxiety has yet to be elucidated.However,the provision of positive images through VR-based visual stimulation may enhance the functionality of brain networks.The salience network is diminished,while the default mode network is enhanced.Additionally,the medial prefrontal cortex may establish a stronger connection with the default mode network,which could result in a reduction of pain and anxiety.Further research into the potential of VR technology to alleviate pain could lead to a reduction in the number of individuals who overdose on painkillers and contribute to positive change in the medical field.

The Analysis of Spatial-Temporal Evolution of City Accessibility Based on Highway Network in Henan Province in China

Accessibility is an important tool to evaluate the maturity of a regional traffic network structure which describes the traffic convenience in the traffic network. The paper defines a new accessibility index by using the resident population weighted average value of the sum of inverse of the traveling time distance and time threshold coming from ordinary traffic network, and then uses this accessibility index to analyze the spatial-temporal characteristics of Henan highway network, as well as its evolution patterns from 2005 to 2020. The results show that with the expansion and improvement of Henan highway network, city accessibility level has been significantly improved, spatial convergence is obvious, the cities in the north central are always High-High aggregation area, the cities in the south are always Low-Low aggregation area, gradually forming the characteristics of Northwest high and Southeast low, relative balance between East and West. There is some non-conforming phenomenon in highway mileage growth and improvement of the city accessibility levels, but this situation is being weakened, the highway network layout is gradually rationalized, the spatial distribution of city accessibility and that of population are beginning to converge.

Spatial-temporal Characteristics of Network Attention of Tengwang Pavilion, a 5A Tourist Attraction in Nanchang City

The temporal evolution and spatial distribution characteristics of network attention of Tengwang Pavilion, a 5 A tourist attraction in Nanchang City from 2011 to 2019 were analyzed by using Baidu index search platform. The results showed that network attention of Tengwang Pavilion in China was increasing year by year, but the annual growth rate was different. There were two peak periods of network attention in a year. They were in April and October respectively. From a weekly point of view, the network attention of Tengwang Pavilion was the lowest on Friday, the highest on Saturdays, and higher on Saturdays and Sundays than on weekdays. From the point of view of geographical distribution, the province that paid the most attention to Tengwang Pavilion on network was Jiangxi Province and the largest city was Nanchang. Tengwang Pavilion scenic spot should pay more attention to the network attention and distribution characteristics of tourists, grasp the potential needs to better guide the development and marketing of tourism products, ensure the safety of tourist attractions, and promote the sustainable development of scenic spots.

Unlocking the future:Mitochondrial genes and neural networks in predicting ovarian cancer prognosis and immunotherapy response

BACKGROUND Mitochondrial genes are involved in tumor metabolism in ovarian cancer(OC)and affect immune cell infiltration and treatment responses.AIM To predict prognosis and immunotherapy response in patients diagnosed with OC using mitochondrial genes and neural networks.METHODS Prognosis,immunotherapy efficacy,and next-generation sequencing data of patients with OC were downloaded from The Cancer Genome Atlas and Gene Expression Omnibus.Mitochondrial genes were sourced from the MitoCarta3.0 database.The discovery cohort for model construction was created from 70% of the patients,whereas the remaining 30% constituted the validation cohort.Using the expression of mitochondrial genes as the predictor variable and based on neural network algorithm,the overall survival time and immunotherapy efficacy(complete or partial response)of patients were predicted.RESULTS In total,375 patients with OC were included to construct the prognostic model,and 26 patients were included to construct the immune efficacy model.The average area under the receiver operating characteristic curve of the prognostic model was 0.7268[95% confidence interval(CI):0.7258-0.7278]in the discovery cohort and 0.6475(95%CI:0.6466-0.6484)in the validation cohort.The average area under the receiver operating characteristic curve of the immunotherapy efficacy model was 0.9444(95%CI:0.8333-1.0000)in the discovery cohort and 0.9167(95%CI:0.6667-1.0000)in the validation cohort.CONCLUSION The application of mitochondrial genes and neural networks has the potential to predict prognosis and immunotherapy response in patients with OC,providing valuable insights into personalized treatment strategies.

Resting-state brain network remodeling after different nerve reconstruction surgeries:a functional magnetic resonance imaging study in brachial plexus injury rats

Distinct brain remodeling has been found after different nerve reconstruction strategies,including motor representation of the affected limb.However,differences among reconstruction strategies at the brain network level have not been elucidated.This study aimed to explore intranetwork changes related to altered peripheral neural pathways after different nerve reconstruction surgeries,including nerve repair,endto-end nerve transfer,and end-to-side nerve transfer.Sprague–Dawley rats underwent complete left brachial plexus transection and were divided into four equal groups of eight:no nerve repair,grafted nerve repair,phrenic nerve end-to-end transfer,and end-to-side transfer with a graft sutured to the anterior upper trunk.Resting-state brain functional magnetic resonance imaging was obtained 7 months after surgery.The independent component analysis algorithm was utilized to identify group-level network components of interest and extract resting-state functional connectivity values of each voxel within the component.Alterations in intra-network resting-state functional connectivity were compared among the groups.Target muscle reinnervation was assessed by behavioral observation(elbow flexion)and electromyography.The results showed that alterations in the sensorimotor and interoception networks were mostly related to changes in the peripheral neural pathway.Nerve repair was related to enhanced connectivity within the sensorimotor network,while end-to-side nerve transfer might be more beneficial for restoring control over the affected limb by the original motor representation.The thalamic-cortical pathway was enhanced within the interoception network after nerve repair and end-to-end nerve transfer.Brain areas related to cognition and emotion were enhanced after end-to-side nerve transfer.Our study revealed important brain networks related to different nerve reconstructions.These networks may be potential targets for enhancing motor recovery.

Spatial-temporal Evolvement Characteristics of Climate Productivity for the Plants on Inner Mongolia Desert Steppe

Thornthwaite Memorial model and other statistic methods were used to calculate the climate-productivity of plants with the meteorological data from 1961 to 2007 at 9 stations distributed on Inner Mongolia desert steppe.The spatial and temporal variation characteristics of climate-productivity were analyzed by using the methods of the tendency rate of the climate trend,accumulative anomaly,and spatial difference and so on.The results showed that the climate-productivity kept linear increased trend over Inner Mongolia desert steppe in recent 47 years,but not significant.In spatial distribution,the climate-productivity reduced with the increased latitude.The climate-productivity in southwest part of Inner Mongolia desert steppe was growing while that in the southeast was reducing.The variation rate of the climate-productivity increased from the northwest part to the southeast part of Inner Mongolia desert steppe.In recent 47 years,the climate-productivity in southeast Jurh underwent the greatest decreasing extent,and the region was the sensitive area of the climate-productivity variation.