Multivariate Time Series Anomaly Detection Based on Spatial-Temporal Network and Transformer in Industrial Internet of Things

In the Industrial Internet of Things(IIoT),sensors generate time series data to reflect the working state.When the systems are attacked,timely identification of outliers in time series is critical to ensure security.Although many anomaly detection methods have been proposed,the temporal correlation of the time series over the same sensor and the state(spatial)correlation between different sensors are rarely considered simultaneously in these methods.Owing to the superior capability of Transformer in learning time series features.This paper proposes a time series anomaly detection method based on a spatial-temporal network and an improved Transformer.Additionally,the methods based on graph neural networks typically include a graph structure learning module and an anomaly detection module,which are interdependent.However,in the initial phase of training,since neither of the modules has reached an optimal state,their performance may influence each other.This scenario makes the end-to-end training approach hard to effectively direct the learning trajectory of each module.This interdependence between the modules,coupled with the initial instability,may cause the model to find it hard to find the optimal solution during the training process,resulting in unsatisfactory results.We introduce an adaptive graph structure learning method to obtain the optimal model parameters and graph structure.Experiments on two publicly available datasets demonstrate that the proposed method attains higher anomaly detection results than other methods.

A step to the decentralized real-time timekeeping network

The composite time scale(CTS) provides an accurate and stable time-frequency reference for modern science and technology. Conventional CTS always features a centralized network topology, which means that the CTS is accompanied by a local master clock. This largely restricts the stability and reliability of the CTS. We simulate the restriction and analyze the influence of the master clock on the CTS. It proves that the CTS's long-term stability is also positively related to that of the master clock, until the region dominated by the frequency drift of the H-maser(averaging time longer than ~10~5s).Aiming at this restriction, a real-time clock network is utilized. Based on the network, a real-time CTS referenced by a stable remote master clock is achieved. The experiment comparing two real-time CTSs referenced by a local and a remote master clock respectively reveals that under open-loop steering, the stability of the CTS is improved by referencing to a remote and more stable master clock instead of a local and less stable master clock. In this way, with the help of the proposed scheme, the CTS can be referenced to the most stable master clock within the network in real time, no matter whether it is local or remote, making democratic polycentric timekeeping possible.

TSCND:Temporal Subsequence-Based Convolutional Network with Difference for Time Series Forecasting

Time series forecasting plays an important role in various fields, such as energy, finance, transport, and weather. Temporal convolutional networks (TCNs) based on dilated causal convolution have been widely used in time series forecasting. However, two problems weaken the performance of TCNs. One is that in dilated casual convolution, causal convolution leads to the receptive fields of outputs being concentrated in the earlier part of the input sequence, whereas the recent input information will be severely lost. The other is that the distribution shift problem in time series has not been adequately solved. To address the first problem, we propose a subsequence-based dilated convolution method (SDC). By using multiple convolutional filters to convolve elements of neighboring subsequences, the method extracts temporal features from a growing receptive field via a growing subsequence rather than a single element. Ultimately, the receptive field of each output element can cover the whole input sequence. To address the second problem, we propose a difference and compensation method (DCM). The method reduces the discrepancies between and within the input sequences by difference operations and then compensates the outputs for the information lost due to difference operations. Based on SDC and DCM, we further construct a temporal subsequence-based convolutional network with difference (TSCND) for time series forecasting. The experimental results show that TSCND can reduce prediction mean squared error by 7.3% and save runtime, compared with state-of-the-art models and vanilla TCN.

Automated Machine Learning Algorithm Using Recurrent Neural Network to Perform Long-Term Time Series Forecasting

Long-term time series forecasting stands as a crucial research domain within the realm of automated machine learning(AutoML).At present,forecasting,whether rooted in machine learning or statistical learning,typically relies on expert input and necessitates substantial manual involvement.This manual effort spans model development,feature engineering,hyper-parameter tuning,and the intricate construction of time series models.The complexity of these tasks renders complete automation unfeasible,as they inherently demand human intervention at multiple junctures.To surmount these challenges,this article proposes leveraging Long Short-Term Memory,which is the variant of Recurrent Neural Networks,harnessing memory cells and gating mechanisms to facilitate long-term time series prediction.However,forecasting accuracy by particular neural network and traditional models can degrade significantly,when addressing long-term time-series tasks.Therefore,our research demonstrates that this innovative approach outperforms the traditional Autoregressive Integrated Moving Average(ARIMA)method in forecasting long-term univariate time series.ARIMA is a high-quality and competitive model in time series prediction,and yet it requires significant preprocessing efforts.Using multiple accuracy metrics,we have evaluated both ARIMA and proposed method on the simulated time-series data and real data in both short and long term.Furthermore,our findings indicate its superiority over alternative network architectures,including Fully Connected Neural Networks,Convolutional Neural Networks,and Nonpooling Convolutional Neural Networks.Our AutoML approach enables non-professional to attain highly accurate and effective time series forecasting,and can be widely applied to various domains,particularly in business and finance.

Hybrid model for BOF oxygen blowing time prediction based on oxygen balance mechanism and deep neural network

The amount of oxygen blown into the converter is one of the key parameters for the control of the converter blowing process,which directly affects the tap-to-tap time of converter. In this study, a hybrid model based on oxygen balance mechanism (OBM) and deep neural network (DNN) was established for predicting oxygen blowing time in converter. A three-step method was utilized in the hybrid model. First, the oxygen consumption volume was predicted by the OBM model and DNN model, respectively. Second, a more accurate oxygen consumption volume was obtained by integrating the OBM model and DNN model. Finally, the converter oxygen blowing time was calculated according to the oxygen consumption volume and the oxygen supply intensity of each heat. The proposed hybrid model was verified using the actual data collected from an integrated steel plant in China, and compared with multiple linear regression model, OBM model, and neural network model including extreme learning machine, back propagation neural network, and DNN. The test results indicate that the hybrid model with a network structure of 3 hidden layer layers, 32-16-8 neurons per hidden layer, and 0.1 learning rate has the best prediction accuracy and stronger generalization ability compared with other models. The predicted hit ratio of oxygen consumption volume within the error±300 m^(3)is 96.67%;determination coefficient (R^(2)) and root mean square error (RMSE) are0.6984 and 150.03 m^(3), respectively. The oxygen blow time prediction hit ratio within the error±0.6 min is 89.50%;R2and RMSE are0.9486 and 0.3592 min, respectively. As a result, the proposed model can effectively predict the oxygen consumption volume and oxygen blowing time in the converter.

Effects of data smoothing and recurrent neural network(RNN)algorithms for real-time forecasting of tunnel boring machine(TBM)performance

Tunnel boring machines(TBMs)have been widely utilised in tunnel construction due to their high efficiency and reliability.Accurately predicting TBM performance can improve project time management,cost control,and risk management.This study aims to use deep learning to develop real-time models for predicting the penetration rate(PR).The models are built using data from the Changsha metro project,and their performances are evaluated using unseen data from the Zhengzhou Metro project.In one-step forecast,the predicted penetration rate follows the trend of the measured penetration rate in both training and testing.The autoregressive integrated moving average(ARIMA)model is compared with the recurrent neural network(RNN)model.The results show that univariate models,which only consider historical penetration rate itself,perform better than multivariate models that take into account multiple geological and operational parameters(GEO and OP).Next,an RNN variant combining time series of penetration rate with the last-step geological and operational parameters is developed,and it performs better than other models.A sensitivity analysis shows that the penetration rate is the most important parameter,while other parameters have a smaller impact on time series forecasting.It is also found that smoothed data are easier to predict with high accuracy.Nevertheless,over-simplified data can lose real characteristics in time series.In conclusion,the RNN variant can accurately predict the next-step penetration rate,and data smoothing is crucial in time series forecasting.This study provides practical guidance for TBM performance forecasting in practical engineering.

基于TimeGAN数据增强的复杂过程故障分类方法

针对传统基于重构的故障分类方法在故障样本稀疏或失衡情况下效果不佳、故障子空间区分能力弱的问题,提出基于TimeGAN数据增强的复杂过程故障分类方法.针对小子样故障,使用TimeGAN对历史故障数据进行数据增强,生成与历史数据分布相似的虚拟故障样本;采用马氏距离评估虚拟样本的质量,剔除不可信样本,构造平衡的故障样本集.将故障样本映射到高维核空间,并在核空间中提取故障子空间.设计故障分类策略并定义4种故障分类性能评估指标以定量衡量算法的分类性能.Tennessee Eastman应用结果表明,所提数据增强方法可以有效扩充故障样本,进而提高故障重构率.与WGAN-GP和SMOTE方法进行对比,发现基于TimeGAN数据增强的故障分类方法具有更好的分类性能.

A NEW METHOD FOR FINDING THE NATURAL FREQUENCY SET OF A LINEAR TIME-INVARIANT NETWORK

提出了一种求线性定常ｎ阶网络的固有频率集的新方法，给出了通用方程的推导及证明．该方法首次将ｎ阶网络的固有频率与ｎ端口网络参数相联系，方程形式简洁对偶，物理意义明确，求解简捷规范且不会产生丢根现象，具有通用性和系统性．

Time Series Forecasting Fusion Network Model Based on Prophet and Improved LSTM

Time series forecasting and analysis are widely used in many fields and application scenarios.Time series historical data reflects the change pattern and trend,which can serve the application and decision in each application scenario to a certain extent.In this paper,we select the time series prediction problem in the atmospheric environment scenario to start the application research.In terms of data support,we obtain the data of nearly 3500 vehicles in some cities in China fromRunwoda Research Institute,focusing on the major pollutant emission data of non-road mobile machinery and high emission vehicles in Beijing and Bozhou,Anhui Province to build the dataset and conduct the time series prediction analysis experiments on them.This paper proposes a P-gLSTNet model,and uses Autoregressive Integrated Moving Average model(ARIMA),long and short-term memory(LSTM),and Prophet to predict and compare the emissions in the future period.The experiments are validated on four public data sets and one self-collected data set,and the mean absolute error(MAE),root mean square error(RMSE),and mean absolute percentage error(MAPE)are selected as the evaluationmetrics.The experimental results show that the proposed P-gLSTNet fusion model predicts less error,outperforms the backbone method,and is more suitable for the prediction of time-series data in this scenario.

Synchronization of stochastic complex networks with time-delayed coupling

Noise and time delay are inevitable in real-world networks. In this article, the framework of master stability function is generalized to stochastic complex networks with time-delayed coupling. The focus is on the effects of noise, time delay,and their inner interactions on the network synchronization. It is found that when there exists time-delayed coupling in the network and noise diffuses through all state variables of nodes, appropriately increasing the noise intensity can effectively improve the network synchronizability;otherwise, noise can be either beneficial or harmful. For stochastic networks, large time delays will lead to desynchronization. These findings provide valuable references for designing optimal complex networks in practical applications.

Short‐time wind speed prediction based on Legendre multi‐wavelet neural network

As one of the most widespread renewable energy sources,wind energy is now an important part of the power system.Accurate and appropriate wind speed forecasting has an essential impact on wind energy utilisation.However,due to the stochastic and un-certain nature of wind energy,more accurate forecasting is necessary for its more stable and safer utilisation.This paper proposes a Legendre multiwavelet‐based neural network model for non‐linear wind speed prediction.It combines the excellent properties of Legendre multi‐wavelets with the self‐learning capability of neural networks,which has rigorous mathematical theory support.It learns input‐output data pairs and shares weights within divided subintervals,which can greatly reduce computing costs.We explore the effectiveness of Legendre multi‐wavelets as an activation function.Mean-while,it is successfully being applied to wind speed prediction.In addition,the appli-cation of Legendre multi‐wavelet neural networks in a hybrid model in decomposition‐reconstruction mode to wind speed prediction problems is also discussed.Numerical results on real data sets show that the proposed model is able to achieve optimal per-formance and high prediction accuracy.In particular,the model shows a more stable performance in multi‐step prediction,illustrating its superiority.

Recent Trends of In-Vehicle Time Sensitive Networking Technologies, Applications and Challenges

With the vigorous development of automobile industry,in-vehicle network is also constantly upgraded to meet data transmission requirements of emerging applications.The main transmission requirements are low latency and certainty especially for autonomous driving.Time sensitive networking(TSN)based on Ethernet gives a possible solution to these requirements.Previous surveys usually investigated TSN from a general perspective,which referred to TSN of various application fields.In this paper,we focus on the application of TSN to the in-vehicle networks.For in-vehicle networks,we discuss all related TSN standards specified by IEEE 802.1 work group up to now.We further overview and analyze recent literature on various aspects of TSN for automotive applications,including synchronization,resource reservation,scheduling,certainty,software and hardware.Application scenarios of TSN for in-vehicle networks are analyzed one by one.Since TSN of in-vehicle network is still at a very initial stage,this paper also gives insights on open issues,future research directions and possible solutions.

Deep learning framework for time series classification based on multiple imaging and hybrid quantum neural networks

Time series classification(TSC)has attracted a lot of attention for time series data mining tasks and has been applied in various fields.With the success of deep learning(DL)in computer vision recognition,people are starting to use deep learning to tackle TSC tasks.Quantum neural networks(QNN)have recently demonstrated their superiority over traditional machine learning in methods such as image processing and natural language processing,but research using quantum neural networks to handle TSC tasks has not received enough attention.Therefore,we proposed a learning framework based on multiple imaging and hybrid QNN(MIHQNN)for TSC tasks.We investigate the possibility of converting 1D time series to 2D images and classifying the converted images using hybrid QNN.We explored the differences between MIHQNN based on single time series imaging and MIHQNN based on the fusion of multiple time series imaging.Four quantum circuits were also selected and designed to study the impact of quantum circuits on TSC tasks.We tested our method on several standard datasets and achieved significant results compared to several current TSC methods,demonstrating the effectiveness of MIHQNN.This research highlights the potential of applying quantum computing to TSC and provides the theoretical and experimental background for future research.

Regional Economic Development Trend Prediction Method Based on Digital Twins and Time Series Network

At present,the interpretation of regional economic development(RED)has changed from a simple evaluation of economic growth to a focus on economic growth and the optimization of economic structure,the improvement of economic relations,and the change of institutional innovation.This article uses the RED trend as the research object and constructs the RED index to conduct the theoretical analysis.Then this paper uses the attention mechanism based on digital twins and the time series network model to verify the actual data.Finally,the regional economy is predicted according to the theoretical model.The specific research work mainly includes the following aspects:1)This paper introduced the development status of research on time series networks and economic forecasting at home and abroad.2)This paper introduces the basic principles and structures of long and short-term memory(LSTM)and convolutional neural network(CNN),constructs an improved CNN-LSTM model combined with the attention mechanism,and then constructs a regional economic prediction index system.3)The best parameters of the model are selected through experiments,and the trained model is used for simulation experiment prediction.The results show that the CNN-LSTM model based on the attentionmechanism proposed in this paper has high accuracy in predicting regional economies.

A consensus time synchronization protocol in wireless sensor network

Time synchronization is one of the base techniques in wireless sensor networks(WSNs).This paper proposes a novel time synchronization protocol which is a robust consensusbased algorithm in the existence of transmission delay and packet loss.It compensates for transmission delay and packet loss firstly,and then,estimates clock skew and clock offset in two steps.Simulation and experiment results show that the proposed protocol can keep synchronization error below 2μs in the grid network of 10 nodes or the random network of 90 nodes.Moreover,the synchronization accuracy in the proposed protocol can keep constant when the WSN works up to a month.

Metaheuristics Based Node Localization Approach for Real-Time Clustered Wireless Networks

In recent times,real time wireless networks have found their applicability in several practical applications such as smart city,healthcare,surveillance,environmental monitoring,etc.At the same time,proper localization of nodes in real time wireless networks helps to improve the overall functioning of networks.This study presents an Improved Metaheuristics based Energy Efficient Clustering with Node Localization(IM-EECNL)approach for real-time wireless networks.The proposed IM-EECNL technique involves two major processes namely node localization and clustering.Firstly,Chaotic Water Strider Algorithm based Node Localization(CWSANL)technique to determine the unknown position of the nodes.Secondly,an Oppositional Archimedes Optimization Algorithm based Clustering(OAOAC)technique is applied to accomplish energy efficiency in the network.Besides,the OAOAC technique derives afitness function comprising residual energy,distance to cluster heads(CHs),distance to base station(BS),and load.The performance validation of the IM-EECNL technique is carried out under several aspects such as localization and energy efficiency.A wide ranging comparative outcomes analysis highlighted the improved performance of the IM-EECNL approach on the recent approaches with the maximum packet delivery ratio(PDR)of 0.985.

Prediction of impedance responses of protonic ceramic cells using artificial neural network tuned with the distribution of relaxation times

A deep-learning-based framework is proposed to predict the impedance response and underlying electrochemical behavior of the reversible protonic ceramic cell(PCC) across a wide variety of different operating conditions.Electrochemical impedance spectra(EIS) of PCCs were first acquired under a variety of opera ting conditions to provide a dataset containing 36 sets of EIS spectra for the model.An artificial neural network(ANN) was then trained to model the relationship between the cell operating condition and EIS response.Finally,ANN model-predicted EIS spectra were analyzed by the distribution of relaxation times(DRT) and compared to DRT spectra obtained from the experimental EIS data,enabling an assessment of the accumulative errors from the predicted EIS data vs the predicted DRT.We show that in certain cases,although the R^(2)of the predicted EIS curve may be> 0.98,the R^(2)of the predicted DRT may be as low as~0.3.This can lead to an inaccurate ANN prediction of the underlying time-resolved electrochemical response,although the apparent accuracy as evaluated from the EIS prediction may seem acceptable.After adjustment of the parameters of the ANN framework,the average R^(2)of the DRTs derived from the predicted EIS can be improved to 0.9667.Thus,we demonstrate that a properly tuned ANN model can be used as an effective tool to predict not only the EIS,but also the DRT of complex electrochemical systems.

A Time Series Intrusion Detection Method Based on SSAE,TCN and Bi-LSTM

In the fast-evolving landscape of digital networks,the incidence of network intrusions has escalated alarmingly.Simultaneously,the crucial role of time series data in intrusion detection remains largely underappreciated,with most systems failing to capture the time-bound nuances of network traffic.This leads to compromised detection accuracy and overlooked temporal patterns.Addressing this gap,we introduce a novel SSAE-TCN-BiLSTM(STL)model that integrates time series analysis,significantly enhancing detection capabilities.Our approach reduces feature dimensionalitywith a Stacked Sparse Autoencoder(SSAE)and extracts temporally relevant features through a Temporal Convolutional Network(TCN)and Bidirectional Long Short-term Memory Network(Bi-LSTM).By meticulously adjusting time steps,we underscore the significance of temporal data in bolstering detection accuracy.On the UNSW-NB15 dataset,ourmodel achieved an F1-score of 99.49%,Accuracy of 99.43%,Precision of 99.38%,Recall of 99.60%,and an inference time of 4.24 s.For the CICDS2017 dataset,we recorded an F1-score of 99.53%,Accuracy of 99.62%,Precision of 99.27%,Recall of 99.79%,and an inference time of 5.72 s.These findings not only confirm the STL model’s superior performance but also its operational efficiency,underpinning its significance in real-world cybersecurity scenarios where rapid response is paramount.Our contribution represents a significant advance in cybersecurity,proposing a model that excels in accuracy and adaptability to the dynamic nature of network traffic,setting a new benchmark for intrusion detection systems.

The Lightweight Edge-Side Fault Diagnosis Approach Based on Spiking Neural Network

Network fault diagnosis methods play a vital role in maintaining network service quality and enhancing user experience as an integral component of intelligent network management.Considering the unique characteristics of edge networks,such as limited resources,complex network faults,and the need for high real-time performance,enhancing and optimizing existing network fault diagnosis methods is necessary.Therefore,this paper proposes the lightweight edge-side fault diagnosis approach based on a spiking neural network(LSNN).Firstly,we use the Izhikevich neurons model to replace the Leaky Integrate and Fire(LIF)neurons model in the LSNN model.Izhikevich neurons inherit the simplicity of LIF neurons but also possess richer behavioral characteristics and flexibility to handle diverse data inputs.Inspired by Fast Spiking Interneurons(FSIs)with a high-frequency firing pattern,we use the parameters of FSIs.Secondly,inspired by the connection mode based on spiking dynamics in the basal ganglia(BG)area of the brain,we propose the pruning approach based on the FSIs of the BG in LSNN to improve computational efficiency and reduce the demand for computing resources and energy consumption.Furthermore,we propose a multiple iterative Dynamic Spike Timing Dependent Plasticity(DSTDP)algorithm to enhance the accuracy of the LSNN model.Experiments on two server fault datasets demonstrate significant precision,recall,and F1 improvements across three diagnosis dimensions.Simultaneously,lightweight indicators such as Params and FLOPs significantly reduced,showcasing the LSNN’s advanced performance and model efficiency.To conclude,experiment results on a pair of datasets indicate that the LSNN model surpasses traditional models and achieves cutting-edge outcomes in network fault diagnosis tasks.

Exploring reservoir computing:Implementation via double stochastic nanowire networks

Neuromorphic computing,inspired by the human brain,uses memristor devices for complex tasks.Recent studies show that self-organizing random nanowires can implement neuromorphic information processing,enabling data analysis.This paper presents a model based on these nanowire networks,with an improved conductance variation profile.We suggest using these networks for temporal information processing via a reservoir computing scheme and propose an efficient data encoding method using voltage pulses.The nanowire network layer generates dynamic behaviors for pulse voltages,allowing time series prediction analysis.Our experiment uses a double stochastic nanowire network architecture for processing multiple input signals,outperforming traditional reservoir computing in terms of fewer nodes,enriched dynamics and improved prediction accuracy.Experimental results confirm the high accuracy of this architecture on multiple real-time series datasets,making neuromorphic nanowire networks promising for physical implementation of reservoir computing.