Multivariate Time Series Anomaly Detection Based on Spatial-Temporal Network and Transformer in Industrial Internet of Things

In the Industrial Internet of Things(IIoT),sensors generate time series data to reflect the working state.When the systems are attacked,timely identification of outliers in time series is critical to ensure security.Although many anomaly detection methods have been proposed,the temporal correlation of the time series over the same sensor and the state(spatial)correlation between different sensors are rarely considered simultaneously in these methods.Owing to the superior capability of Transformer in learning time series features.This paper proposes a time series anomaly detection method based on a spatial-temporal network and an improved Transformer.Additionally,the methods based on graph neural networks typically include a graph structure learning module and an anomaly detection module,which are interdependent.However,in the initial phase of training,since neither of the modules has reached an optimal state,their performance may influence each other.This scenario makes the end-to-end training approach hard to effectively direct the learning trajectory of each module.This interdependence between the modules,coupled with the initial instability,may cause the model to find it hard to find the optimal solution during the training process,resulting in unsatisfactory results.We introduce an adaptive graph structure learning method to obtain the optimal model parameters and graph structure.Experiments on two publicly available datasets demonstrate that the proposed method attains higher anomaly detection results than other methods.

Fine-Grained Multivariate Time Series Anomaly Detection in IoT

Sensors produce a large amount of multivariate time series data to record the states of Internet of Things(IoT)systems.Multivariate time series timestamp anomaly detection(TSAD)can identify timestamps of attacks and malfunctions.However,it is necessary to determine which sensor or indicator is abnormal to facilitate a more detailed diagnosis,a process referred to as fine-grained anomaly detection(FGAD).Although further FGAD can be extended based on TSAD methods,existing works do not provide a quantitative evaluation,and the performance is unknown.Therefore,to tackle the FGAD problem,this paper first verifies that the TSAD methods achieve low performance when applied to the FGAD task directly because of the excessive fusion of features and the ignoring of the relationship’s dynamic changes between indicators.Accordingly,this paper proposes a mul-tivariate time series fine-grained anomaly detection(MFGAD)framework.To avoid excessive fusion of features,MFGAD constructs two sub-models to independently identify the abnormal timestamp and abnormal indicator instead of a single model and then combines the two kinds of abnormal results to detect the fine-grained anomaly.Based on this framework,an algorithm based on Graph Attention Neural Network(GAT)and Attention Convolutional Long-Short Term Memory(A-ConvLSTM)is proposed,in which GAT learns temporal features of multiple indicators to detect abnormal timestamps and A-ConvLSTM captures the dynamic relationship between indicators to identify abnormal indicators.Extensive simulations on a real-world dataset demonstrate that the proposed algorithm can achieve a higher F1 score and hit rate than the extension of existing TSAD methods with the benefit of two independent sub-models for timestamp and indicator detection.

Designing Artemisinins with Antimalarial Potential, Combining Molecular Electrostatic Potential, Ligand-Heme Interaction and Multivariate Models

Artemisinins tested against W-2 strains of malaria falciparum are investigated with molecular electrostatic potential (MEP), in an attempt to identify key features of the compounds that are necessary for their activities, as well as to investigate likely interactions with the receptor in a biological process and to use that information to propose new molecules. In order to discover the best geometry involving the ligand-receptor complexes (heme) studied and help in the proposition of the new derivatives, molecular simulations of interactions between the most negative charged region around the peroxide and heme locates (the ones around the Fe2+ ion) were carried out. In addition, PCA (principal components analysis), HCA (hierarchical cluster analysis), SDA (stepwise discriminant analysis), and KNN (K-nearest neighbor) multivariate models were employed to investigate which descriptors are responsible for the classification between the higher and lower antimalarial activity of the compounds, and also this information was used to propose new potentially active molecules. The information accumulated in studies of MEP, molecular docking, and multivariate analysis supported the proposal of new structures with potential antimalarial activities. The multivariate models constructed were applied to the new structures and indicated numbers 19 and 20 as the most prominent for syntheses and biological assays.

A Memory-Guided Anomaly Detection Model with Contrastive Learning for Multivariate Time Series

Some reconstruction-based anomaly detection models in multivariate time series have brought impressive performance advancements but suffer from weak generalization ability and a lack of anomaly identification.These limitations can result in the misjudgment of models,leading to a degradation in overall detection performance.This paper proposes a novel transformer-like anomaly detection model adopting a contrastive learning module and a memory block(CLME)to overcome the above limitations.The contrastive learning module tailored for time series data can learn the contextual relationships to generate temporal fine-grained representations.The memory block can record normal patterns of these representations through the utilization of attention-based addressing and reintegration mechanisms.These two modules together effectively alleviate the problem of generalization.Furthermore,this paper introduces a fusion anomaly detection strategy that comprehensively takes into account the residual and feature spaces.Such a strategy can enlarge the discrepancies between normal and abnormal data,which is more conducive to anomaly identification.The proposed CLME model not only efficiently enhances the generalization performance but also improves the ability of anomaly detection.To validate the efficacy of the proposed approach,extensive experiments are conducted on well-established benchmark datasets,including SWaT,PSM,WADI,and MSL.The results demonstrate outstanding performance,with F1 scores of 90.58%,94.83%,91.58%,and 91.75%,respectively.These findings affirm the superiority of the CLME model over existing stateof-the-art anomaly detection methodologies in terms of its ability to detect anomalies within complex datasets accurately.

Seasonal Variability of Biofouling Community Structure in the Gulf of Mannar,Southeast Coast of India:A Multivariate Approach

In this research,an Underwater Biofouling Panel(UWBFP)system was erected for the qualitative and quantitative estimation of macro fouling organisms in the Gulf of Mannar.Forty-four biofoulers were identified from four types of selected test panels.Among these biofoulers,Amphibalanus amphitrite(Darwin,1854)was the dominant one.The concrete panel encouraged the highest barnacle density compared to the other panels.Three series of test panels were used to assess the seasonal density of biofouling communities.The overall variation in barnacle count in the seaward and shoreward sides of all these three series were tested.They were found to be significantly different from each other.The greater variations in the barnacle density observed in this study in A-series of test panels could be due to the lack of or absence of other foulers to compete within the fortnight.The Shannon-Wiener species diversity index showed the highest diversity in wood substratum among the three series with greater accumulation of different types of fouling organisms.Multivariate analyses were also performed to understand the seasonal variation as well as the settlement pattern on the different directions of test panels based on validated data.PCA showed a strong variability(PC1 between 70.8%and 98.6%variance)between the directions of the panels in connection with barnacle density.The shade plot and CAP analysis segregated the short-term A-series test panels from the long-term(B-and C-series)test panels.Hence,the output was helpful in understanding the recruitment status of various faunal resources involved in the biofouling processes.

Mapping species assemblages of tropical forests at different hierarchical levels based on multivariate regression trees

Background: Vegetation distribution maps are of great significance for nature protection and management. In diverse tropical forests, accurate spatial mapping of vegetation types is challenging;the high species diversity and abundance of rare species challenge classification concepts, while remote sensing signals may not vary systematically with species composition, complicating the technical capability for delineating vegetation types in the landscape.Methods: We used a combination of field-based compositional data and their relations to environmental variables to predict the distribution of forest types in the Wuzhishan National Natural Reserve(WNNR), Hainan Island,China, using multivariate regression trees(MRT). The MRT was based on arboreal vegetation composition in 132plots of 20 m×20 m with a regular spacing of 1 km. Apart from the MRT, non-metric multidimensional scaling(NMDS) was used to evaluate vegetation-environment relationships.Results: The MRT model worked best when using 14 key environmental variables including topography, climate,latitude and soil, although the difference with the simpler model including only topographical variables was small. The full model classified the 132 plots into 3 vegetation types, 6 formation groups, 20 formations and 65associations at different hierarchical syntaxonomic levels. This model was the basis for forest vegetation maps for the WNNR. MRT and NMDS showed that elevation was the main driving force for the distribution of vegetation types and formation groups. Climate, latitude, and soil(especially available P), together with topographic variables, all influenced the distribution of formations and associations.Conclusions: While elevation determines forest-type distributions, lower-level syntaxonomic forest classes respond to the topographic diversity typical for mountains. Apart from providing the first detailed forest vegetation map for any part of WNNR, we show how, in spite of limitations, MRT with existing environmental data can be a useful method for mapping diverse and remote tropical forests.

Detection of healthy and pathological heartbeat dynamics in ECG signals using multivariate recurrence networks with multiple scale factors

The electrocardiogram(ECG)is one of the physiological signals applied in medical clinics to determine health status.The physiological complexity of the cardiac system is related to age,disease,etc.For the investigation of the effects of age and cardiovascular disease on the cardiac system,we then construct multivariate recurrence networks with multiple scale factors from multivariate time series.We propose a new concept of cross-clustering coefficient entropy to construct a weighted network,and calculate the average weighted path length and the graph energy of the weighted network to quantitatively probe the topological properties.The obtained results suggest that these two network measures show distinct changes between different subjects.This is because,with aging or cardiovascular disease,a reduction in the conductivity or structural changes in the myocardium of the heart contributes to a reduction in the complexity of the cardiac system.Consequently,the complexity of the cardiac system is reduced.After that,the support vector machine(SVM)classifier is adopted to evaluate the performance of the proposed approach.Accuracy of 94.1%and 95.58%between healthy and myocardial infarction is achieved on two datasets.Therefore,this method can be adopted for the development of a noninvasive and low-cost clinical prognostic system to identify heart-related diseases and detect hidden state changes in the cardiac system.

Proposal and Pilot Study: A Generalization of the W or W'Statistic for Multivariate Normality

The aim of this paper is to present a generalization of the Shapiro-Wilk W-test or Shapiro-Francia W'-test for application to two or more variables. It consists of calculating all the unweighted linear combinations of the variables and their W- or W'-statistics with the Royston’s log-transformation and standardization, zln(1-W) or zln(1-W'). Because the calculation of the probability of zln(1-W) or zln(1-W') is to the right tail, negative values are truncated to 0 before doing their sum of squares. Independence in the sequence of these half-normally distributed values is required for the test statistic to follow a chi-square distribution. This assumption is checked using the robust Ljung-Box test. One degree of freedom is lost for each cancelled value. Defined the new test with its two variants (Q-test or Q'-test), 50 random samples with 4 variables and 20 participants were generated, 20% following a multivariate normal distribution and 80% deviating from this distribution. The new test was compared with Mardia’s, runs, and Royston’s tests. Central tendency differences in type II error and statistical power were tested using the Friedman’s test and pairwise comparisons using the Wilcoxon’s test. Differences in the frequency of successes in statistical decision making were compared using the Cochran’s Q test and pairwise comparisons using the McNemar’s test. Sensitivity, specificity and efficiency proportions were compared using the McNemar’s Z test. The generated 50 samples were classified into five ordered categories of deviation from multivariate normality, the correlation between this variable and p-value of each test was calculated using the Spearman’s coefficient and these correlations were compared. Family-wise error rate corrections were applied. The new test and the Royston’s test were the best choices, with a very slight advantage Q-test over Q'-test. Based on these promising results, further study and use of this new sensitive, specific and effective test are suggested.

Graph Construction Method for GNN-Based Multivariate Time-Series Forecasting

Multivariate time-series forecasting(MTSF)plays an important role in diverse real-world applications.To achieve better accuracy in MTSF,time-series patterns in each variable and interrelationship patterns between variables should be considered together.Recently,graph neural networks(GNNs)has gained much attention as they can learn both patterns using a graph.For accurate forecasting through GNN,a well-defined graph is required.However,existing GNNs have limitations in reflecting the spectral similarity and time delay between nodes,and consider all nodes with the same weight when constructing graph.In this paper,we propose a novel graph construction method that solves aforementioned limitations.We first calculate the Fourier transform-based spectral similarity and then update this similarity to reflect the time delay.Then,we weight each node according to the number of edge connections to get the final graph and utilize it to train the GNN model.Through experiments on various datasets,we demonstrated that the proposed method enhanced the performance of GNN-based MTSF models,and the proposed forecasting model achieve of up to 18.1%predictive performance improvement over the state-of-the-art model.

Dynamic Ensemble Multivariate Time Series Forecasting Model for PM2.5

In forecasting real time environmental factors,large data is needed to analyse the pattern behind the data values.Air pollution is a major threat towards developing countries and it is proliferating every year.Many methods in time ser-ies prediction and deep learning models to estimate the severity of air pollution.Each independent variable contributing towards pollution is necessary to analyse the trend behind the air pollution in that particular locality.This approach selects multivariate time series and coalesce a real time updatable autoregressive model to forecast Particulate matter(PM)PM2.5.To perform experimental analysis the data from the Central Pollution Control Board(CPCB)is used.Prediction is car-ried out for Chennai with seven locations and estimated PM’s using the weighted ensemble method.Proposed method for air pollution prediction unveiled effective and moored performance in long term prediction.Dynamic budge with high weighted k-models are used simultaneously and devising an ensemble helps to achieve stable forecasting.Computational time of ensemble decreases with paral-lel processing in each sub model.Weighted ensemble model shows high perfor-mance in long term prediction when compared to the traditional time series models like Vector Auto-Regression(VAR),Autoregressive Integrated with Mov-ing Average(ARIMA),Autoregressive Moving Average with Extended terms(ARMEX).Evaluation metrics like Root Mean Square Error(RMSE),Mean Absolute Error(MAE)and the time to achieve the time series are compared.

Multivariate Aggregated NOMA for Resource Aware Wireless Network Communication Security

NonorthogonalMultiple Access(NOMA)is incorporated into the wireless network systems to achieve better connectivity,spectral and energy effectiveness,higher data transfer rate,and also obtain the high quality of services(QoS).In order to improve throughput and minimum latency,aMultivariate Renkonen Regressive Weighted Preference Bootstrap Aggregation based Nonorthogonal Multiple Access(MRRWPBA-NOMA)technique is introduced for network communication.In the downlink transmission,each mobile device’s resources and their characteristics like energy,bandwidth,and trust are measured.Followed by,the Weighted Preference Bootstrap Aggregation is applied to recognize the resource-efficient mobile devices for aware data transmission by constructing the different weak hypotheses i.e.,Multivariate Renkonen Regression functions.Based on the classification,resource and trust-aware devices are selected for transmission.Simulation of the proposed MRRWPBA-NOMA technique and existing methods are carried out with different metrics such as data delivery ratio,throughput,latency,packet loss rate,and energy efficiency,signaling overhead.The simulation results assessment indicates that the proposed MRRWPBA-NOMA outperforms well than the conventional methods.

An improved bidirectional generative adversarial network model for multivariate estimation of correlated and imbalanced tunnel construction parameters

Estimation of construction parameters is crucial for optimizing tunnel construction schedule.Due to the influence of routine activities and occasional risk events,these parameters are usually correlated and imbalanced.To solve this issue,an improved bidirectional generative adversarial network(BiGAN)model with a joint discriminator structure and zero-centered gradient penalty(0-GP)is proposed.In this model,in order to improve the capability of original BiGAN in learning imbalanced parameters,the joint discriminator separately discriminates the routine activities and risk event durations to balance their influence weights.Then,the self-attention mechanism is embedded so that the discriminator can pay more attention to the imbalanced parameters.Finally,the 0-GP is adapted for the loss of the discrimi-nator to improve its convergence and stability.A case study of a tunnel in China shows that the improved BiGAN can obtain parameter estimates consistent with the classical Gauss mixture model,without the need of tedious and complex correlation analysis.The proposed joint discriminator can increase the ability of BiGAN in estimating imbalanced construction parameters,and the 0-GP can ensure the stability and convergence of the model.

Application of Multivariate Reinforcement Learning Engine in Optimizing the Power Generation Process of Domestic Waste Incineration

Garbage incineration is an ideal method for the harmless and resource-oriented treatment of urban domestic waste.However,current domestic waste incineration power plants often face challenges related to maintaining consistent steam production and high operational costs.This article capitalizes on the technical advantages of big data artificial intelligence,optimizing the power generation process of domestic waste incineration as the entry point,and adopts four main engine modules of Alibaba Cloud reinforcement learning algorithm engine,operating parameter prediction engine,anomaly recognition engine,and video visual recognition algorithm engine.The reinforcement learning algorithm extracts the operational parameters of each incinerator to obtain a control benchmark.Through the operating parameter prediction algorithm,prediction models for drum pressure,primary steam flow,NOx,SO2,and HCl are constructed to achieve short-term prediction of operational parameters,ultimately improving control performance.The anomaly recognition algorithm develops a thickness identification model for the material layer in the drying section,allowing for rapid and effective assessment of feed material thickness to ensure uniformity control.Meanwhile,the visual recognition algorithm identifies flame images and assesses the combustion status and location of the combustion fire line within the furnace.This real-time understanding of furnace flame combustion conditions guides adjustments to the grate and air volume.Integrating AI technology into the waste incineration sector empowers the environmental protection industry with the potential to leverage big data.This development holds practical significance in optimizing the harmless and resource-oriented treatment of urban domestic waste,reducing operational costs,and increasing efficiency.

Tool Condition Monitoring Based on Nonlinear Output Frequency Response Functions and Multivariate Control Chart

Tool condition monitoring(TCM)is a key technology for intelligent manufacturing.The objective is to monitor the tool operation status and detect tool breakage so that the tool can be changed in time to avoid significant damage to workpieces and reduce manufacturing costs.Recently,an innovative TCM approach based on sensor data modelling and model frequency analysis has been proposed.Different from traditional signal feature-based monitoring,the data from sensors are utilized to build a dynamic process model.Then,the nonlinear output frequency response functions,a concept which extends the linear system frequency response function to the nonlinear case,over the frequency range of the tooth passing frequency of the machining process are extracted to reveal tool health conditions.In order to extend the novel sensor data modelling and model frequency analysis to unsupervised condition monitoring of cutting tools,in the present study,a multivariate control chart is proposed for TCM based on the frequency domain properties of machining processes derived from the innovative sensor data modelling and model frequency analysis.The feature dimension is reduced by principal component analysis first.Then the moving average strategy is exploited to generate monitoring variables and overcome the effects of noises.The milling experiments of titanium alloys are conducted to verify the effectiveness of the proposed approach in detecting excessive flank wear of solid carbide end mills.The results demonstrate the advantages of the new approach over conventional TCM techniques and its potential in industrial applications.

Joint multivariate statistical model and its applications to synthetic earthquake predic-tion

针对目前地震综合预报中的一些问题，利用近30年来迅速发展的多元统计分析中主成分分析、判别分析组成多元统计组合模型，在众多的地震预报指标(预报因子)中采用信息最大化方法，选择对中期预测信息累积贡献率大于90％地震预报指标，分别进行相关分析、预测、检验，最终应用马氏距离判别作外推综合预报；并以华北地区(30°～42°N，108°125°E)为例进行模型的应用检验，初步研究已取得了较好的效果．

多元质量特性预报：MULTIVARIATE回归分析的应用

对现象之间客观存在的因果关系建立回归分析模型,这是实际中较为普遍的做法.在这篇文章中,我们根据MULTIVARIATE回归分析的基本原理,利用从生产现场采集的观测数据,对产品两个质量特性及其五个关键影响因素之间的关系建立了多重多元回归分析方程,为说明MULTIVARIATE回归应用的可行性,我们还结合实例给出了因变量向量估计的两种形式,以及无条件预报的置信区间。

Outcomes of treatment of male urethral stricture：a multivariate analysis

目的：分析外伤性和前列腺术后尿道狭窄各种治疗方法的优缺点及影响因素，为临床上合理选择治疗方式、减少狭窄复发提出有益建议。方法：对本科64例外伤性和59例前列腺术后的尿道狭窄初次治疗共123例进行回顾性多因素分析。结果：64例外伤性尿道狭窄患者中，尿扩22例，20例（90．9％）复发；尿道内切开21例，16例（76．2％）复发；尿道端端吻合21例，4例（19％）复发；59例前列腺术后尿道狭窄中，尿扩16例，15例（93．6％）复发；尿道内切开37例，5例（13．5％）复发；6例切开膀胱行膀胱颈疤痕切开切除膀胱颈整形术，3例（50％）复发。结论：①经尿道疤痕切开切除治疗外伤性尿道狭窄，其疗效与狭窄长度有关，狭窄长度〈2cm复发率低，〉2121／1则复发率高。②尿道疤痕切除端端吻合治疗外伤性尿道狭窄，其疗效与狭窄长度、狭窄部位、既往手术史无关，与手术本身有关，即术中如彻底切除狭窄疤痕及坏死组织、吻合无张力则复发率低，反之则高。⑧尿扩适用于尿道黏膜下狭窄，不适用于合并有尿道海绵体纤维化的尿道狭窄。④尿道内切开是治疗前列腺术后尿道狭窄的首选方法且疗效好。

Study on QSAR of Taxol and its Derivatives Based on Stepwise Multivariate Linear Regression Analysis

应用逐步多元线性回归方法，对紫杉醇及其衍生物的两个异构体系进行了构效关系研究，发现Ｃ１３位侧链中的Ｃ３＇位取代基的摩尔折射与其活性密切相关，从而可以推断出Ｃ３′位取代基的改变很可能是其生物活性变化的重要因素，这为设计与合成活性较高的紫杉醇类似物将提供有价值的参考信息。

Multivariate adaptive regression splines and neural network models for prediction of pile drivability

Piles are long, slender structural elements used to transfer the loads from the superstructure through weak strata onto stiffer soils or rocks. For driven piles, the impact of the piling hammer induces compression and tension stresses in the piles. Hence, an important design consideration is to check that the strength of the pile is sufficient to resist the stresses caused by the impact of the pile hammer. Due to its complexity, pile drivability lacks a precise analytical solution with regard to the phenomena involved.In situations where measured data or numerical hypothetical results are available, neural networks stand out in mapping the nonlinear interactions and relationships between the system’s predictors and dependent responses. In addition, unlike most computational tools, no mathematical relationship assumption between the dependent and independent variables has to be made. Nevertheless, neural networks have been criticized for their long trial-and-error training process since the optimal configuration is not known a priori. This paper investigates the use of a fairly simple nonparametric regression algorithm known as multivariate adaptive regression splines（MARS）, as an alternative to neural networks, to approximate the relationship between the inputs and dependent response, and to mathematically interpret the relationship between the various parameters. In this paper, the Back propagation neural network（BPNN） and MARS models are developed for assessing pile drivability in relation to the prediction of the Maximum compressive stresses（MCS）, Maximum tensile stresses（MTS）, and Blow per foot（BPF）. A database of more than four thousand piles is utilized for model development and comparative performance between BPNN and MARS predictions.

GIS-based landslide susceptibility mapping using numerical risk factor bivariate model and its ensemble with linear multivariate regression and boosted regression tree algorithms

In this study, a novel approach of the landslide numerical risk factor(LNRF) bivariate model was used in ensemble with linear multivariate regression(LMR) and boosted regression tree(BRT) models, coupled with radar remote sensing data and geographic information system(GIS), for landslide susceptibility mapping(LSM) in the Gorganroud watershed, Iran. Fifteen topographic, hydrological, geological and environmental conditioning factors and a landslide inventory(70%, or 298 landslides) were used in mapping. Phased array-type L-band synthetic aperture radar data were used to extract topographic parameters. Coefficients of tolerance and variance inflation factor were used to determine the coherence among conditioning factors. Data for the landslide inventory map were obtained from various resources, such as Iranian Landslide Working Party(ILWP), Forestry, Rangeland and Watershed Organisation(FRWO), extensive field surveys, interpretation of aerial photos and satellite images, and radar data. Of the total data, 30% were used to validate LSMs, using area under the curve(AUC), frequency ratio(FR) and seed cell area index(SCAI).Normalised difference vegetation index, land use/land cover and slope degree in BRT model elevation, rainfall and distance from stream were found to be important factors and were given the highest weightage in modelling. Validation results using AUC showed that the ensemble LNRF-BRT and LNRFLMR models(AUC = 0.912(91.2%) and 0.907(90.7%), respectively) had high predictive accuracy than the LNRF model alone(AUC = 0.855(85.5%)). The FR and SCAI analyses showed that all models divided the parameter classes with high precision. Overall, our novel approach of combining multivariate and machine learning methods with bivariate models, radar remote sensing data and GIS proved to be a powerful tool for landslide susceptibility mapping.