Hierarchical multihead self-attention for time-series-based fault diagnosis

Fault diagnosis is important for maintaining the safety and effectiveness of chemical process.Considering the multivariate,nonlinear,and dynamic characteristic of chemical process,many time-series-based data-driven fault diagnosis methods have been developed in recent years.However,the existing methods have the problem of long-term dependency and are difficult to train due to the sequential way of training.To overcome these problems,a novel fault diagnosis method based on time-series and the hierarchical multihead self-attention(HMSAN)is proposed for chemical process.First,a sliding window strategy is adopted to construct the normalized time-series dataset.Second,the HMSAN is developed to extract the time-relevant features from the time-series process data.It improves the basic self-attention model in both width and depth.With the multihead structure,the HMSAN can pay attention to different aspects of the complicated chemical process and obtain the global dynamic features.However,the multiple heads in parallel lead to redundant information,which cannot improve the diagnosis performance.With the hierarchical structure,the redundant information is reduced and the deep local time-related features are further extracted.Besides,a novel many-to-one training strategy is introduced for HMSAN to simplify the training procedure and capture the long-term dependency.Finally,the effectiveness of the proposed method is demonstrated by two chemical cases.The experimental results show that the proposed method achieves a great performance on time-series industrial data and outperforms the state-of-the-art approaches.

Modeling urban redevelopment:A novel approach using time-series remote sensing data and machine learning

Accurate mapping and timely monitoring of urban redevelopment are pivotal for urban studies and decisionmakers to foster sustainable urban development.Traditional mapping methods heavily depend on field surveys and subjective questionnaires,yielding less objective,reliable,and timely data.Recent advancements in Geographic Information Systems(GIS)and remote-sensing technologies have improved the identification and mapping of urban redevelopment through quantitative analysis using satellite-based observations.Nonetheless,challenges persist,particularly concerning accuracy and significant temporal delays.This study introduces a novel approach to modeling urban redevelopment,leveraging machine learning algorithms and remote-sensing data.This methodology can facilitate the accurate and timely identification of urban redevelopment activities.The study’s machine learning model can analyze time-series remote-sensing data to identify spatio-temporal and spectral patterns related to urban redevelopment.The model is thoroughly evaluated,and the results indicate that it can accurately capture the time-series patterns of urban redevelopment.This research’s findings are useful for evaluating urban demographic and economic changes,informing policymaking and urban planning,and contributing to sustainable urban development.The model can also serve as a foundation for future research on early-stage urban redevelopment detection and evaluation of the causes and impacts of urban redevelopment.

Missing Value Imputation for Radar-Derived Time-Series Tracks of Aerial Targets Based on Improved Self-Attention-Based Network

The frequent missing values in radar-derived time-series tracks of aerial targets(RTT-AT)lead to significant challenges in subsequent data-driven tasks.However,the majority of imputation research focuses on random missing(RM)that differs significantly from common missing patterns of RTT-AT.The method for solving the RM may experience performance degradation or failure when applied to RTT-AT imputation.Conventional autoregressive deep learning methods are prone to error accumulation and long-term dependency loss.In this paper,a non-autoregressive imputation model that addresses the issue of missing value imputation for two common missing patterns in RTT-AT is proposed.Our model consists of two probabilistic sparse diagonal masking self-attention(PSDMSA)units and a weight fusion unit.It learns missing values by combining the representations outputted by the two units,aiming to minimize the difference between the missing values and their actual values.The PSDMSA units effectively capture temporal dependencies and attribute correlations between time steps,improving imputation quality.The weight fusion unit automatically updates the weights of the output representations from the two units to obtain a more accurate final representation.The experimental results indicate that,despite varying missing rates in the two missing patterns,our model consistently outperforms other methods in imputation performance and exhibits a low frequency of deviations in estimates for specific missing entries.Compared to the state-of-the-art autoregressive deep learning imputation model Bidirectional Recurrent Imputation for Time Series(BRITS),our proposed model reduces mean absolute error(MAE)by 31%~50%.Additionally,the model attains a training speed that is 4 to 8 times faster when compared to both BRITS and a standard Transformer model when trained on the same dataset.Finally,the findings from the ablation experiments demonstrate that the PSDMSA,the weight fusion unit,cascade network design,and imputation loss enhance imputation performance and confirm the efficacy of our design.

Detecting winter canola(Brassica napus) phenological stages using an improved shape-model method based on time-series UAV spectral data

Accurate information about phenological stages is essential for canola field management practices such as irrigation, fertilization, and harvesting. Previous studies in canola phenology monitoring focused mainly on the flowering stage, using its apparent structure features and colors. Additional phenological stages have been largely overlooked. The objective of this study was to improve a shape-model method(SMM) for extracting winter canola phenological stages from time-series top-of-canopy reflectance images collected by an unmanned aerial vehicle(UAV). The transformation equation of the SMM was refined to account for the multi-peak features of the temporal dynamics of three vegetation indices(VIs)(NDVI, EVI, and CI). An experiment with various seeding scenarios was conducted, including four different seeding dates and three seeding densities. Three mathematical functions: asymmetric Gaussian function(AGF), Fourier function, and double logistic function, were employed to fit timeseries vegetation indices to extract information about phenological stages. The refined SMM effectively estimated the phenological stages of canola, with a minimum root mean square error(RMSE) of 3.7 days for all phenological stages. The AGF function provided the best fitting performance, as it captured multiple peaks in the growth dynamics characteristics for all seeding date scenarios using four scaling parameters. For the three selected VIs, CIred-edgeachieved the greatest accuracy in estimating the phenological stage dates. This study demonstrates the high potential of the refined SMM for estimating winter canola phenology.

A Simple Additive Weighting Method for Time-SeriesMultiindices Decision Making and Its Applications

This paper takes the evaluation of overall economic benefit by an example and proposes a simple additive weighting method for time-series multiindices decision making. The method can automatically determine the weight coefficients among the multiindices and the years respectively and it also can obtain the objective evaluation results and conclusions.

Research on Stock Price Prediction Method Based on the GAN-LSTM-Attention Model

Stock price prediction is a typical complex time series prediction problem characterized by dynamics,nonlinearity,and complexity.This paper introduces a generative adversarial network model that incorporates an attention mechanism(GAN-LSTM-Attention)to improve the accuracy of stock price prediction.Firstly,the generator of this model combines the Long and Short-Term Memory Network(LSTM),the Attention Mechanism and,the Fully-Connected Layer,focusing on generating the predicted stock price.The discriminator combines the Convolutional Neural Network(CNN)and the Fully-Connected Layer to discriminate between real stock prices and generated stock prices.Secondly,to evaluate the practical application ability and generalization ability of the GAN-LSTM-Attention model,four representative stocks in the United States of America(USA)stock market,namely,Standard&Poor’s 500 Index stock,Apple Incorporatedstock,AdvancedMicroDevices Incorporatedstock,and Google Incorporated stock were selected for prediction experiments,and the prediction performance was comprehensively evaluated by using the three evaluation metrics,namely,mean absolute error(MAE),root mean square error(RMSE),and coefficient of determination(R2).Finally,the specific effects of the attention mechanism,convolutional layer,and fully-connected layer on the prediction performance of the model are systematically analyzed through ablation study.The results of experiment show that the GAN-LSTM-Attention model exhibits excellent performance and robustness in stock price prediction.

Graph Construction Method for GNN-Based Multivariate Time-Series Forecasting

Multivariate time-series forecasting(MTSF)plays an important role in diverse real-world applications.To achieve better accuracy in MTSF,time-series patterns in each variable and interrelationship patterns between variables should be considered together.Recently,graph neural networks(GNNs)has gained much attention as they can learn both patterns using a graph.For accurate forecasting through GNN,a well-defined graph is required.However,existing GNNs have limitations in reflecting the spectral similarity and time delay between nodes,and consider all nodes with the same weight when constructing graph.In this paper,we propose a novel graph construction method that solves aforementioned limitations.We first calculate the Fourier transform-based spectral similarity and then update this similarity to reflect the time delay.Then,we weight each node according to the number of edge connections to get the final graph and utilize it to train the GNN model.Through experiments on various datasets,we demonstrated that the proposed method enhanced the performance of GNN-based MTSF models,and the proposed forecasting model achieve of up to 18.1%predictive performance improvement over the state-of-the-art model.

Study of the Transport Behavior of Multispherical Proppant in Intersecting Fracture Based on Discrete Element Method

To analyze the differences in the transport and distribution of different types of proppants and to address issues such as the short effective support of proppant and poor placement in hydraulically intersecting fractures,this study considered the combined impact of geological-engineering factors on conductivity.Using reservoir production parameters and the discrete elementmethod,multispherical proppants were constructed.Additionally,a 3D fracture model,based on the specified conditions of the L block,employed coupled(Computational Fluid Dynamics)CFD-DEM(Discrete ElementMethod)for joint simulations to quantitatively analyze the transport and placement patterns of multispherical proppants in intersecting fractures.Results indicate that turbulent kinetic energy is an intrinsic factor affecting proppant transport.Moreover,the efficiency of placement and migration distance of low-sphericity quartz sand constructed by the DEM in the main fracture are significantly reduced compared to spherical ceramic proppants,with a 27.7%decrease in the volume fraction of the fracture surface,subsequently affecting the placement concentration and damaging fracture conductivity.Compared to small-angle fractures,controlling artificial and natural fractures to expand at angles of 45°to 60°increases the effective support length by approximately 20.6%.During hydraulic fracturing of gas wells,ensuring the fracture support area and post-closure conductivity can be achieved by controlling the sphericity of proppants and adjusting the perforation direction to control the direction of artificial fractures.

Optimal Scheduling of an Independent Electro-Hydrogen System with Hybrid Energy Storage Using a Multi-Objective Standardization Fusion Method

In the independent electro-hydrogen system(IEHS)with hybrid energy storage(HESS),achieving optimal scheduling is crucial.Still,it presents a challenge due to the significant deviations in values ofmultiple optimization objective functions caused by their physical dimensions.These deviations seriously affect the scheduling process.A novel standardization fusion method has been established to address this issue by analyzing the variation process of each objective function’s values.The optimal scheduling results of IEHS with HESS indicate that the economy and overall energy loss can be improved 2–3 times under different optimization methods.The proposed method better balances all optimization objective functions and reduces the impact of their dimensionality.When the cost of BESS decreases by approximately 30%,its participation deepens by about 1 time.Moreover,if the price of the electrolyzer is less than 15￥/kWh or if the cost of the fuel cell drops below 4￥/kWh,their participation will increase substantially.This study aims to provide a more reasonable approach to solving multi-objective optimization problems.

Structural Modal Parameter Recognition and Related Damage Identification Methods under Environmental Excitations: A Review

Modal parameters can accurately characterize the structural dynamic properties and assess the physical state of the structure.Therefore,it is particularly significant to identify the structural modal parameters according to the monitoring data information in the structural health monitoring(SHM)system,so as to provide a scientific basis for structural damage identification and dynamic model modification.In view of this,this paper reviews methods for identifying structural modal parameters under environmental excitation and briefly describes how to identify structural damages based on the derived modal parameters.The paper primarily introduces data-driven modal parameter recognition methods(e.g.,time-domain,frequency-domain,and time-frequency-domain methods,etc.),briefly describes damage identification methods based on the variations of modal parameters(e.g.,natural frequency,modal shapes,and curvature modal shapes,etc.)and modal validation methods(e.g.,Stability Diagram and Modal Assurance Criterion,etc.).The current status of the application of artificial intelligence(AI)methods in the direction of modal parameter recognition and damage identification is further discussed.Based on the pre-vious analysis,the main development trends of structural modal parameter recognition and damage identification methods are given to provide scientific references for the optimized design and functional upgrading of SHM systems.

Temperature and Daily Mortality in Shanghai:A Time-series Study

To investigate the association between temperature and daily mortality in Shanghai from June 1, 2000 to December 31, 2001. Methods Time-series approach was used to estimate the effect of temperature on daily total and cause-specific mortality. We fitted generalized additive Poisson regression using non-parametric smooth functions to control for long-term time trend, season and other variables. We also controlled for day of the week. Results A gently sloping V-like relationship between total mortality and temperature was found, with an optimum temperature (e.g. temperature with lowest mortality risk) value of 26.7癈 in Shanghai. For temperatures above the optimum value, total mortality increased by 0.73% for each degree Celsius increase; while for temperature below the optimum value, total mortality decreased by 1.21% for each degree Celsius increase. Conclusions Our findings indicate that temperature has an effect on daily mortality in Shanghai, and the time-series approach is a useful tool for studying the temperature-mortality association.

Review of the SBAS InSAR Time-series algorithms, applications, and challenges

In the past 30 years,the small baseline subset(SBAS)InSAR time-series technique has emerged as an essential tool for measuring slow surface displacement and estimating geophysical parameters.Because of its ability to monitor large-scale deformation with millimeter accuracy,the SBAS method has been widely used in various geodetic fields,such as ground subsidence,landslides,and seismic activity.The obtained long-term time-series cumulative deformation is vital for studying the deformation mecha-nism.This article reviews the algorithms,applications,and challenges of the SBAS method.First,we recall the fundamental principle and analyze the shortcomings of the traditional SBAS algorithm,which provides a basic framework for the following improved time series methods.Second,we classify the current improved SBAS techniques from different perspectives:solving the ill-posed equation,increasing the density of high-coherence points,improving the accuracy of monitoring deformation and measuring the multi-dimensional deformation.Third,we summarize the application of the SBAS method in monitoring ground subsidence,permafrost degradation,glacier movement,volcanic activity,landslides,and seismic activity.Finally,we discuss the difficulties faced by the SBAS method and explore its future development direction.

Mapping winter wheat using phenological feature of peak before winter on the North China Plain based on time-series MODIS data

By employing the unique phenological feature of winter wheat extracted from peak before winter （PBW） and the advantages of moderate resolution imaging spectroradiometer （MODIS） data with high temporal resolution and intermediate spatial resolution, a remote sensing-based model for mapping winter wheat on the North China Plain was built through integration with Landsat images and land-use data. First, a phenological window, PBW was drawn from time-series MODIS data. Next, feature extraction was performed for the PBW to reduce feature dimension and enhance its information. Finally, a regression model was built to model the relationship of the phenological feature and the sample data. The amount of information of the PBW was evaluated and compared with that of the main peak （MP）. The relative precision of the mapping reached up to 92% in comparison to the Landsat sample data, and ranged between 87 and 96% in comparison to the statistical data. These results were sufficient to satisfy the accuracy requirements for winter wheat mapping at a large scale. Moreover, the proposed method has the ability to obtain the distribution information for winter wheat in an earlier period than previous studies. This study could throw light on the monitoring of winter wheat in China by using unique phenological feature of winter wheat.

Clustering Structure Analysis in Time-Series Data With Density-Based Clusterability Measure

Clustering is used to gain an intuition of the struc tures in the data.Most of the current clustering algorithms pro duce a clustering structure even on data that do not possess such structure.In these cases,the algorithms force a structure in the data instead of discovering one.To avoid false structures in the relations of data,a novel clusterability assessment method called density-based clusterability measure is proposed in this paper.I measures the prominence of clustering structure in the data to evaluate whether a cluster analysis could produce a meaningfu insight to the relationships in the data.This is especially useful in time-series data since visualizing the structure in time-series data is hard.The performance of the clusterability measure is evalu ated against several synthetic data sets and time-series data sets which illustrate that the density-based clusterability measure can successfully indicate clustering structure of time-series data.

Wavelet matrix transform for time-series similarity measurement

A time-series similarity measurement method based on wavelet and matrix transform was proposed,and its anti-noise ability,sensitivity and accuracy were discussed. The time-series sequences were compressed into wavelet subspace,and sample feature vector and orthogonal basics of sample time-series sequences were obtained by K-L transform. Then the inner product transform was carried out to project analyzed time-series sequence into orthogonal basics to gain analyzed feature vectors. The similarity was calculated between sample feature vector and analyzed feature vector by the Euclid distance. Taking fault wave of power electronic devices for example,the experimental results show that the proposed method has low dimension of feature vector,the anti-noise ability of proposed method is 30 times as large as that of plain wavelet method,the sensitivity of proposed method is 1/3 as large as that of plain wavelet method,and the accuracy of proposed method is higher than that of the wavelet singular value decomposition method. The proposed method can be applied in similarity matching and indexing for lager time series databases.

Time-series gas prediction model using LS-SVR within a Bayesian framework

The traditional least squares support vector regression（LS-SVR）model,using cross validation to determine the regularization parameter and kernel parameter,is time-consuming.We propose a Bayesian evidence framework to infer the LS-SVR model parameters.Three levels Bayesian inferences are used to determine the model parameters,regularization hyper-parameters and tune the nuclear parameters by model comparison.On this basis,we established Bayesian LS-SVR time-series gas forecasting models and provide steps for the algorithm.The gas outburst data of a Hebi 10th mine working face is used to validate the model.The optimal embedding dimension and delay time of the time series were obtained by the smallest differential entropy method.Finally,within a MATLAB7.1 environment,we used actual coal gas data to compare the traditional LS-SVR and the Bayesian LS-SVR with LS-SVMlab1.5 Toolbox simulation.The results show that the Bayesian framework of an LS-SVR significantly improves the speed and accuracy of the forecast.

Spatio-temporal changes of underground coal fires during 2008-2016 in Khanh Hoa coal field(North-east of Viet Nam) using Landsat time-series data

Underground coal fires are one of the most common and serious geohazards in most coal producing countries in the world. Monitoring their spatio-temporal changes plays an important role in controlling and preventing the effects of coal fires, and their environmental impact. In this study, the spatio-temporal changes of underground coal fires in Khanh Hoa coal field(North-East of Viet Nam) were analyzed using Landsat time-series data during the 2008-2016 period. Based on land surface temperatures retrieved from Landsat thermal data, underground coal fires related to thermal anomalies were identified using the MEDIAN+1.5×IQR(IQR: Interquartile range) threshold technique. The locations of underground coal fires were validated using a coal fire map produced by the field survey data and cross-validated using the daytime ASTER thermal infrared imagery. Based on the fires extracted from seven Landsat thermal imageries, the spatiotemporal changes of underground coal fire areas were analyzed. The results showed that the thermalanomalous zones have been correlated with known coal fires. Cross-validation of coal fires using ASTER TIR data showed a high consistency of 79.3%. The largest coal fire area of 184.6 hectares was detected in 2010, followed by 2014(181.1 hectares) and 2016(178.5 hectares). The smaller coal fire areas were extracted with areas of 133.6 and 152.5 hectares in 2011 and 2009 respectively. Underground coal fires were mainly detected in the northern and southern part, and tend to spread to north-west of the coal field.

Time-series gene expression prof iles in AGS cells stimulated with Helicobacter pylori

AIM: To extend the knowledge of the dynamic interaction between Helicobacter pylori (H. pylori) and host mucosa. METHODS: A time-series cDNA microarray was performed in order to detect the temporal gene expression prof iles of human gastric epithelial adenocarcinoma cells infected with H. pylori. Six time points were selected to observe the changes in the model. A differential expression prof ile at each time point was obtained by comparing the microarray signal value with that of 0 h. Real-time polymerase chain reaction was subsequently performed to evaluate the data quality. RESULTS: We found a diversity of gene expression patterns at different time points and identifi ed a group of genes whose expression levels were significantly correlated with several important immune response and tumor related pathways. CONCLUSION: Early infection may trigger some important pathways and may impact the outcome of the infection.

Time-Series Modeling and Prediction of Global Monthly Absolute Temperature for Environmental Decision Making

A generalized, structural, time series modeling framework was developed to analyze the monthly records of absolute surface temperature, one of the most important environmental parameters, using a deterministicstochastic combined （DSC） approach. Although the development of the framework was based on the characterization of the variation patterns of a global dataset, the methodology could be applied to any monthly absolute temperature record. Deterministic processes were used to characterize the variation patterns of the global trend and the cyclic oscillations of the temperature signal, involving polynomial functions and the Fourier method, respectively, while stochastic processes were employed to account for any remaining patterns in the temperature signal, involving seasonal autoregressive integrated moving average （SARIMA） models. A prediction of the monthly global surface temperature during the second decade of the 21st century using the DSC model shows that the global temperature will likely continue to rise at twice the average rate of the past 150 years. The evaluation of prediction accuracy shows that DSC models perform systematically well against selected models of other authors, suggesting that DSC models, when coupled with other ecoenvironmental models, can be used as a supplemental tool for short-term （10-year） environmental planning and decision making.