Multivariate Time Series Anomaly Detection Based on Spatial-Temporal Network and Transformer in Industrial Internet of Things

In the Industrial Internet of Things(IIoT),sensors generate time series data to reflect the working state.When the systems are attacked,timely identification of outliers in time series is critical to ensure security.Although many anomaly detection methods have been proposed,the temporal correlation of the time series over the same sensor and the state(spatial)correlation between different sensors are rarely considered simultaneously in these methods.Owing to the superior capability of Transformer in learning time series features.This paper proposes a time series anomaly detection method based on a spatial-temporal network and an improved Transformer.Additionally,the methods based on graph neural networks typically include a graph structure learning module and an anomaly detection module,which are interdependent.However,in the initial phase of training,since neither of the modules has reached an optimal state,their performance may influence each other.This scenario makes the end-to-end training approach hard to effectively direct the learning trajectory of each module.This interdependence between the modules,coupled with the initial instability,may cause the model to find it hard to find the optimal solution during the training process,resulting in unsatisfactory results.We introduce an adaptive graph structure learning method to obtain the optimal model parameters and graph structure.Experiments on two publicly available datasets demonstrate that the proposed method attains higher anomaly detection results than other methods.

Comprehensive evaluation and spatial-temporal evolution characteristics of urban resilience in Chengdu-Chongqing Economic Circle

To clarify the connotations and extensions of urban resilience,this study focuses on the Chengdu-Chongqing Economic Circle with 16 cities as research subjects.A comprehensive evaluation index system was constructed to measure the resilience of each city from 2003 to 2020.The spatial-temporal evolution characteristics were analyzed using Kernel density estimation,standard deviation ellipse,and spatial Markov chain analysis,and the spatial Tobit model was introduced to discover the influencing factors.The results indicate the following:①Urban resilience in the Chengdu-Chongqing Economic Circle displays an upward trend,with the center of gravity moving to the southwest,and the polarization phenomenon intensifying.②The urban resilience level in a region has certain spatial and geographical dependence,while the probability of urban resilience transfer differs in adjacent cities with different resilience levels.③Urban centrality,economic scale,openness level,and financial development promote urban resilience,whereas government scale significantly inhibits it.Finally,this paper proposes countermeasures and suggestions to improve the urban resilience of the Chengdu-Chongqing Economic Circle.

AFSTGCN:Prediction for multivariate time series using an adaptive fused spatial-temporal graph convolutional network

The prediction for Multivariate Time Series(MTS)explores the interrelationships among variables at historical moments,extracts their relevant characteristics,and is widely used in finance,weather,complex industries and other fields.Furthermore,it is important to construct a digital twin system.However,existing methods do not take full advantage of the potential properties of variables,which results in poor predicted accuracy.In this paper,we propose the Adaptive Fused Spatial-Temporal Graph Convolutional Network(AFSTGCN).First,to address the problem of the unknown spatial-temporal structure,we construct the Adaptive Fused Spatial-Temporal Graph(AFSTG)layer.Specifically,we fuse the spatial-temporal graph based on the interrelationship of spatial graphs.Simultaneously,we construct the adaptive adjacency matrix of the spatial-temporal graph using node embedding methods.Subsequently,to overcome the insufficient extraction of disordered correlation features,we construct the Adaptive Fused Spatial-Temporal Graph Convolutional(AFSTGC)module.The module forces the reordering of disordered temporal,spatial and spatial-temporal dependencies into rule-like data.AFSTGCN dynamically and synchronously acquires potential temporal,spatial and spatial-temporal correlations,thereby fully extracting rich hierarchical feature information to enhance the predicted accuracy.Experiments on different types of MTS datasets demonstrate that the model achieves state-of-the-art single-step and multi-step performance compared with eight other deep learning models.

Spatial-temporal distribution and geochemistry of highly evolved Mesozoic granites in Great Xing’an Range,NE China:Discriminant criteria and geological significance

Highly evolved granite is an important sign of the mature continent crust and closely associated with deposits of rare metals.In this work,the authors undertake systematically zircon U-Pb ages and whole rock elemental data for highly evolved granitic intrusions from the Great Xing’an Range(GXR),NE China,to elucidate their discriminant criteria,spatial-temporal distribution,differentiation and geodynamic mecha-nism.Geochemical data of these highly evolved granites suggest that high w(SiO_(2))(>70%)and differentiation index(DI>88)could be quantified indicators,while strong Eu depletion,high TE_(1,3),lowΣREE and low Zr/Hf,Nb/Ta,K/Rb could only be qualitative indicators.Zircon U-Pb ages suggest that the highly evolved gran-ites in the GXR were mainly formed in Late Mesozoic,which can be divided into two major stages:Late Ju-rassic-early Early Cretaceous(162-136 Ma,peak at 138 Ma),and late Early Cretaceous(136-106 Ma,peak at 126 Ma).The highly evolved granites are mainly distributed in the central-southern GXR,and display a weakly trend of getting younger from northwest to southeast,meanwhile indicating the metallogenic potential of rare metals within the central GXR.The spatial-temporal distribution,combined with regional geological data,indicates the highly evolved Mesozoic granites in the GXR were emplaced in an extensional environ-ment,of which the Late Jurassic-early Early Cretaceous extension was related to the closure of the Mongol-Okhotsk Ocean and roll-back of the Paleo-Pacific Plate,while the late Early Cretaceous extension was mainly related to the roll-back of the Paleo-Pacific Plate.

Spatial-temporal Variation Characteristics of Water Quality in the Lower Reaches of the Nenjiang River

As an important river in the western part of Jilin Province,the lower reach of the Nenjiang River is an important wetland water source conservation area in Jilin Province.Within the watershed,it governs the Momoge Wetland,the Xianghai Wetland,and the Danjiang Wetland in Jilin Province.The main problem in the lower reaches of the Nenjiang River is the uneven distribution of water resources in time and space,and the intensification of land salinization.Zhenlai County and Da an City in the Nenjiang River Basin have sufficient surface water resources,with surface water as the drinking water source.Baicheng City and Tongyu County have scarce surface water resources,and both use groundwater as their domestic water source.The main polluted section in the basin is the Xianghai Reservoir,and the annual water quality evaluation is Class V.However,the water quality of the Tao er River,the main stream of the Nenjiang River,is significantly better than that of the Xianghai Reservoir.In order to better study the water environmental pollution situation in the Nenjiang River basin,monitoring data from five sections of non seasonal rivers in the basin from 2012 to 2021 were selected for studying water quality.This in-depth exploration of the water pollution status and river water quality change trends in the Nenjiang River basin is of great significance for future rural development,agricultural pattern transformation,and the promotion of water ecological civilization construction.

Spatial-temporal difference between nitrate in groundwater and nitrogen in soil based on geostatistical analysis

The study of temporal and spatial variations of nitrate in groundwater under different soil nitrogen environments is helpful to the security of groundwater resources in agricultural areas.In this paper,based on 320 groups of soil and groundwater samples collected at the same time,geostatistical analysis and multiple regression analysis were comprehensively used to conduct the evaluation of nitrogen contents in both groundwater and soil.From May to August,as the nitrification of groundwater is dominant,the average concentration of nitrate nitrogen is 34.80 mg/L;The variation of soil ammonia nitrogen and nitrate nitrogen is moderate from May to July,and the variation coefficient decreased sharply and then increased in August.There is a high correlation between the nitrate nitrogen in groundwater and soil in July,and there is a high correlation between the nitrate nitrogen in groundwater and ammonium nitrogen in soil in August and nitrate nitrogen in soil in July.From May to August,the area of low groundwater nitrate nitrogen in 0-5 mg/L and 5-10 mg/L decreased from 10.97%to 0,and the proportion of high-value area(greater than 70 mg/L)increased from 21.19%to 27.29%.Nitrate nitrogen is the main factor affecting the quality of groundwater.The correlation analysis of nitrate nitrogen in groundwater,nitrate nitrogen in soil and ammonium nitrogen shows that they have a certain period of delay.The areas with high concentration of nitrate in groundwater are mainly concentrated in the western part of the study area,which has a high consistency with the high value areas of soil nitrate distribution from July to August,and a high difference with the spatial position of soil ammonia nitrogen distribution in August.

STGSA:A Novel Spatial-Temporal Graph Synchronous Aggregation Model for Traffic Prediction

The success of intelligent transportation systems relies heavily on accurate traffic prediction,in which how to model the underlying spatial-temporal information from traffic data has come under the spotlight.Most existing frameworks typically utilize separate modules for spatial and temporal correlations modeling.However,this stepwise pattern may limit the effectiveness and efficiency in spatial-temporal feature extraction and cause the overlook of important information in some steps.Furthermore,it is lacking sufficient guidance from prior information while modeling based on a given spatial adjacency graph(e.g.,deriving from the geodesic distance or approximate connectivity),and may not reflect the actual interaction between nodes.To overcome those limitations,our paper proposes a spatial-temporal graph synchronous aggregation(STGSA)model to extract the localized and long-term spatial-temporal dependencies simultaneously.Specifically,a tailored graph aggregation method in the vertex domain is designed to extract spatial and temporal features in one graph convolution process.In each STGSA block,we devise a directed temporal correlation graph to represent the localized and long-term dependencies between nodes,and the potential temporal dependence is further fine-tuned by an adaptive weighting operation.Meanwhile,we construct an elaborated spatial adjacency matrix to represent the road sensor graph by considering both physical distance and node similarity in a datadriven manner.Then,inspired by the multi-head attention mechanism which can jointly emphasize information from different r epresentation subspaces,we construct a multi-stream module based on the STGSA blocks to capture global information.It projects the embedding input repeatedly with multiple different channels.Finally,the predicted values are generated by stacking several multi-stream modules.Extensive experiments are constructed on six real-world datasets,and numerical results show that the proposed STGSA model significantly outperforms the benchmarks.

Dense Spatial-Temporal Graph Convolutional Network Based on Lightweight OpenPose for Detecting Falls

Fall behavior is closely related to high mortality in the elderly,so fall detection becomes an important and urgent research area.However,the existing fall detection methods are difficult to be applied in daily life due to a large amount of calculation and poor detection accuracy.To solve the above problems,this paper proposes a dense spatial-temporal graph convolutional network based on lightweight OpenPose.Lightweight OpenPose uses MobileNet as a feature extraction network,and the prediction layer uses bottleneck-asymmetric structure,thus reducing the amount of the network.The bottleneck-asymmetrical structure compresses the number of input channels of feature maps by 1×1 convolution and replaces the 7×7 convolution structure with the asymmetric structure of 1×7 convolution,7×1 convolution,and 7×7 convolution in parallel.The spatial-temporal graph convolutional network divides the multi-layer convolution into dense blocks,and the convolutional layers in each dense block are connected,thus improving the feature transitivity,enhancing the network’s ability to extract features,thus improving the detection accuracy.Two representative datasets,Multiple Cameras Fall dataset(MCF),and Nanyang Technological University Red Green Blue+Depth Action Recognition dataset(NTU RGB+D),are selected for our experiments,among which NTU RGB+D has two evaluation benchmarks.The results show that the proposed model is superior to the current fall detection models.The accuracy of this network on the MCF dataset is 96.3%,and the accuracies on the two evaluation benchmarks of the NTU RGB+D dataset are 85.6%and 93.5%,respectively.

Spatial-temporal variations and driving factors of soil organic carbon in forest ecosystems of Northeast China

Forest soil carbon is a major carbon pool of terrestrial ecosystems,and accurate estimation of soil organic carbon(SOC)stocks in forest ecosystems is rather challenging.This study compared the prediction performance of three empirical model approaches namely,regression kriging(RK),multiple stepwise regression(MSR),random forest(RF),and boosted regression trees(BRT)to predict SOC stocks in Northeast China for 1990 and 2015.Furthermore,the spatial variation of SOC stocks and the main controlling environmental factors during the past 25 years were identified.A total of 82(in 1990)and 157(in 2015)topsoil(0–20 cm)samples with 12 environmental factors(soil property,climate,topography and biology)were selected for model construction.Randomly selected80%of the soil sample data were used to train the models and the other 20%data for model verification using mean absolute error,root mean square error,coefficient of determination and Lin's consistency correlation coefficient indices.We found BRT model as the best prediction model and it could explain 67%and 60%spatial variation of SOC stocks,in 1990,and 2015,respectively.Predicted maps of all models in both periods showed similar spatial distribution characteristics,with the lower SOC in northeast and higher SOC in southwest.Mean annual temperature and elevation were the key environmental factors influencing the spatial variation of SOC stock in both periods.SOC stocks were mainly stored under Cambosols,Gleyosols and Isohumosols,accounting for 95.6%(1990)and 95.9%(2015).Overall,SOC stocks increased by 471 Tg C during the past 25 years.Our study found that the BRT model employing common environmental factors was the most robust method for forest topsoil SOC stocks inventories.The spatial resolution of BRT model enabled us to pinpoint in which areas of Northeast China that new forest tree planting would be most effective for enhancing forest C stocks.Overall,our approach is likely to be useful in forestry management and ecological restoration at and beyond the regional scale.

Grid System Analysis of Urban Flora of Bukhara City (Uzbekistan)

This article presents information on the study of the flora of Uzbekistan based on grid system mapping. The urban flora of the city of Bukhara was researched in it. As a result of research, the territory of Bukhara city was divided into 85 indexes based on 1 × 1 km2 grid mapping system. The diversity and density of species in the indexes are determined. The influence of anthropogenic factors on the diversity of species in the indexes is determined.

RoGRUT: A Hybrid Deep Learning Model for Detecting Power Trapping in Smart Grids

Electricity theft is a widespread non-technical issue that has a negative impact on both power grids and electricity users.It hinders the economic growth of utility companies,poses electrical risks,and impacts the high energy costs borne by consumers.The development of smart grids is crucial for the identification of power theft since these systems create enormous amounts of data,including information on client consumption,which may be used to identify electricity theft using machine learning and deep learning techniques.Moreover,there also exist different solutions such as hardware-based solutions to detect electricity theft that may require human resources and expensive hardware.Computer-based solutions are presented in the literature to identify electricity theft but due to the dimensionality curse,class imbalance issue and improper hyper-parameter tuning of such models lead to poor performance.In this research,a hybrid deep learning model abbreviated as RoGRUT is proposed to detect electricity theft as amalicious and non-malicious activity.The key steps of the RoGRUT are data preprocessing that covers the problem of class imbalance,feature extraction and final theft detection.Different advanced-level models like RoBERTa is used to address the curse of dimensionality issue,the near miss for class imbalance,and transfer learning for classification.The effectiveness of the RoGRUTis evaluated using the dataset fromactual smartmeters.A significant number of simulations demonstrate that,when compared to its competitors,the RoGRUT achieves the best classification results.The performance evaluation of the proposed model revealed exemplary results across variousmetrics.The accuracy achieved was 88%,with precision at an impressive 86%and recall reaching 84%.The F1-Score,a measure of overall performance,stood at 85%.Furthermore,themodel exhibited a noteworthyMatthew correlation coefficient of 78%and excelled with an area under the curve of 91%.

A Long-Time-Step-Permitting Tracer Transport Model on the Regular Latitude–Longitude Grid

If an explicit time scheme is used in a numerical model, the size of the integration time step is typically limited by the spatial resolution. This study develops a regular latitude–longitude grid-based global three-dimensional tracer transport model that is computationally stable at large time-step sizes. The tracer model employs a finite-volume flux-form semiLagrangian transport scheme in the horizontal and an adaptively implicit algorithm in the vertical. The horizontal and vertical solvers are coupled via a straightforward operator-splitting technique. Both the finite-volume scheme's onedimensional slope-limiter and the adaptively implicit vertical solver's first-order upwind scheme enforce monotonicity. The tracer model permits a large time-step size and is inherently conservative and monotonic. Idealized advection test cases demonstrate that the three-dimensional transport model performs very well in terms of accuracy, stability, and efficiency. It is possible to use this robust transport model in a global atmospheric dynamical core.

Physics-Constrained Robustness Enhancement for Tree Ensembles Applied in Smart Grid

With the widespread use of machine learning(ML)technology,the operational efficiency and responsiveness of power grids have been significantly enhanced,allowing smart grids to achieve high levels of automation and intelligence.However,tree ensemble models commonly used in smart grids are vulnerable to adversarial attacks,making it urgent to enhance their robustness.To address this,we propose a robustness enhancement method that incorporates physical constraints into the node-splitting decisions of tree ensembles.Our algorithm improves robustness by developing a dataset of adversarial examples that comply with physical laws,ensuring training data accurately reflects possible attack scenarios while adhering to physical rules.In our experiments,the proposed method increased robustness against adversarial attacks by 100%when applied to real grid data under physical constraints.These results highlight the advantages of our method in maintaining efficient and secure operation of smart grids under adversarial conditions.

Study on Joint Method of 3D Acoustic Emission Source Localization Simplex and Grid Search Scanning

Acoustic emission(AE)source localization is a fundamental element of rock fracture damage imaging.To improve the efficiency and accuracy of AE source localization,this paper proposes a joint method comprising a three-dimensional(3D)AE source localization simplex method and grid search scanning.Using the concept of the geometry of simplexes,tetrahedral iterations were first conducted to narrow down the suspected source region.This is followed by a process of meshing the region and node searching to scan for optimal solutions,until the source location is determined.The resulting algorithm was tested using the artificial excitation source localization and uniaxial compression tests,after which the localization results were compared with the simplex and exhaustive methods.The results revealed that the localization obtained using the proposed method is more stable and can be effectively avoided compared with the simplex localization method.Furthermore,compared with the global scanning method,the proposed method is more efficient,with an average time of 10%–20%of the global scanning localization algorithm.Thus,the proposed algorithm is of great significance for laboratory research focused on locating rupture damages sustained by large-sized rock masses or test blocks.

Assessment of Crop Yield in China Simulated by Thirteen Global Gridded Crop Models

Global gridded crop models(GGCMs) have been broadly applied to assess the impacts of climate and environmental change and adaptation on agricultural production. China is a major grain producing country, but thus far only a few studies have assessed the performance of GGCMs in China, and these studies mainly focused on the average and interannual variability of national and regional yields. Here, a systematic national-and provincial-scale evaluation of the simulations by13 GGCMs [12 from the GGCM Intercomparison(GGCMI) project, phase 1, and CLM5-crop] of the yields of four crops(wheat, maize, rice, and soybean) in China during 1980–2009 was carried out through comparison with crop yield statistics collected from the National Bureau of Statistics of China. Results showed that GGCMI models generally underestimate the national yield of rice but overestimate it for the other three crops, while CLM5-crop can reproduce the national yields of wheat, maize, and rice well. Most GGCMs struggle to simulate the spatial patterns of crop yields. In terms of temporal variability, GGCMI models generally fail to capture the observed significant increases, but some can skillfully simulate the interannual variability. Conversely, CLM5-crop can represent the increases in wheat, maize, and rice, but works less well in simulating the interannual variability. At least one model can skillfully reproduce the temporal variability of yields in the top-10 producing provinces in China, albeit with a few exceptions. This study, for the first time, provides a complete picture of GGCM performance in China, which is important for GGCM development and understanding the reliability and uncertainty of national-and provincial-scale crop yield prediction in China.

Novel cyber-physical collaborative detection and localization method against dynamic load altering attacks in smart energy grids

Owing to the integration of energy digitization and artificial intelligence technology,smart energy grids can realize the stable,efficient and clean operation of power systems.However,the emergence of cyber-physical attacks,such as dynamic load-altering attacks(DLAAs)has introduced great challenges to the security of smart energy grids.Thus,this study developed a novel cyber-physical collaborative security framework for DLAAs in smart energy grids.The proposed framework integrates attack prediction in the cyber layer with the detection and localization of attacks in the physical layer.First,a data-driven method was proposed to predict the DLAA sequence in the cyber layer.By designing a double radial basis function network,the influence of disturbances on attack prediction can be eliminated.Based on the prediction results,an unknown input observer-based detection and localization method was further developed for the physical layer.In addition,an adaptive threshold was designed to replace the traditional precomputed threshold and improve the detection performance of the DLAAs.Consequently,through the collaborative work of the cyber-physics layer,injected DLAAs were effectively detected and located.Compared with existing methodologies,the simulation results on IEEE 14-bus and 118-bus power systems verified the superiority of the proposed cyber-physical collaborative detection and localization against DLAAs.

Virtual Power Plants for Grid Resilience: A Concise Overview of Research and Applications

The power grid is undergoing a transformation from synchronous generators(SGs) toward inverter-based resources(IBRs). The stochasticity, asynchronicity, and limited-inertia characteristics of IBRs bring about challenges to grid resilience. Virtual power plants(VPPs) are emerging technologies to improve the grid resilience and advance the transformation. By judiciously aggregating geographically distributed energy resources(DERs) as individual electrical entities, VPPs can provide capacity and ancillary services to grid operations and participate in electricity wholesale markets. This paper aims to provide a concise overview of the concept and development of VPPs and the latest progresses in VPP operation, with the focus on VPP scheduling and control. Based on this overview, we identify a few potential challenges in VPP operation and discuss the opportunities of integrating the multi-agent system(MAS)-based strategy into the VPP operation to enhance its scalability, performance and resilience.

Intelligent Power Grid Load Transferring Based on Safe Action-Correction Reinforcement Learning

When a line failure occurs in a power grid, a load transfer is implemented to reconfigure the network by changingthe states of tie-switches and load demands. Computation speed is one of the major performance indicators inpower grid load transfer, as a fast load transfer model can greatly reduce the economic loss of post-fault powergrids. In this study, a reinforcement learning method is developed based on a deep deterministic policy gradient.The tedious training process of the reinforcement learning model can be conducted offline, so the model showssatisfactory performance in real-time operation, indicating that it is suitable for fast load transfer. Consideringthat the reinforcement learning model performs poorly in satisfying safety constraints, a safe action-correctionframework is proposed to modify the learning model. In the framework, the action of load shedding is correctedaccording to sensitivity analysis results under a small discrete increment so as to match the constraints of line flowlimits. The results of case studies indicate that the proposed method is practical for fast and safe power grid loadtransfer.

A U-Shaped Network-Based Grid Tagging Model for Chinese Named Entity Recognition

Chinese named entity recognition(CNER)has received widespread attention as an important task of Chinese information extraction.Most previous research has focused on individually studying flat CNER,overlapped CNER,or discontinuous CNER.However,a unified CNER is often needed in real-world scenarios.Recent studies have shown that grid tagging-based methods based on character-pair relationship classification hold great potential for achieving unified NER.Nevertheless,how to enrich Chinese character-pair grid representations and capture deeper dependencies between character pairs to improve entity recognition performance remains an unresolved challenge.In this study,we enhance the character-pair grid representation by incorporating both local and global information.Significantly,we introduce a new approach by considering the character-pair grid representation matrix as a specialized image,converting the classification of character-pair relationships into a pixel-level semantic segmentation task.We devise a U-shaped network to extract multi-scale and deeper semantic information from the grid image,allowing for a more comprehensive understanding of associative features between character pairs.This approach leads to improved accuracy in predicting their relationships,ultimately enhancing entity recognition performance.We conducted experiments on two public CNER datasets in the biomedical domain,namely CMeEE-V2 and Diakg.The results demonstrate the effectiveness of our approach,which achieves F1-score improvements of 7.29 percentage points and 1.64 percentage points compared to the current state-of-the-art(SOTA)models,respectively.

Prediction of three-dimensional ocean temperature in the South China Sea based on time series gridded data and a dynamic spatiotemporal graph neural network

Ocean temperature is an important physical variable in marine ecosystems,and ocean temperature prediction is an important research objective in ocean-related fields.Currently,one of the commonly used methods for ocean temperature prediction is based on data-driven,but research on this method is mostly limited to the sea surface,with few studies on the prediction of internal ocean temperature.Existing graph neural network-based methods usually use predefined graphs or learned static graphs,which cannot capture the dynamic associations among data.In this study,we propose a novel dynamic spatiotemporal graph neural network(DSTGN)to predict threedimensional ocean temperature(3D-OT),which combines static graph learning and dynamic graph learning to automatically mine two unknown dependencies between sequences based on the original 3D-OT data without prior knowledge.Temporal and spatial dependencies in the time series were then captured using temporal and graph convolutions.We also integrated dynamic graph learning,static graph learning,graph convolution,and temporal convolution into an end-to-end framework for 3D-OT prediction using time-series grid data.In this study,we conducted prediction experiments using high-resolution 3D-OT from the Copernicus global ocean physical reanalysis,with data covering the vertical variation of temperature from the sea surface to 1000 m below the sea surface.We compared five mainstream models that are commonly used for ocean temperature prediction,and the results showed that the method achieved the best prediction results at all prediction scales.