Hierarchical multihead self-attention for time-series-based fault diagnosis

Fault diagnosis is important for maintaining the safety and effectiveness of chemical process.Considering the multivariate,nonlinear,and dynamic characteristic of chemical process,many time-series-based data-driven fault diagnosis methods have been developed in recent years.However,the existing methods have the problem of long-term dependency and are difficult to train due to the sequential way of training.To overcome these problems,a novel fault diagnosis method based on time-series and the hierarchical multihead self-attention(HMSAN)is proposed for chemical process.First,a sliding window strategy is adopted to construct the normalized time-series dataset.Second,the HMSAN is developed to extract the time-relevant features from the time-series process data.It improves the basic self-attention model in both width and depth.With the multihead structure,the HMSAN can pay attention to different aspects of the complicated chemical process and obtain the global dynamic features.However,the multiple heads in parallel lead to redundant information,which cannot improve the diagnosis performance.With the hierarchical structure,the redundant information is reduced and the deep local time-related features are further extracted.Besides,a novel many-to-one training strategy is introduced for HMSAN to simplify the training procedure and capture the long-term dependency.Finally,the effectiveness of the proposed method is demonstrated by two chemical cases.The experimental results show that the proposed method achieves a great performance on time-series industrial data and outperforms the state-of-the-art approaches.

Modeling urban redevelopment:A novel approach using time-series remote sensing data and machine learning

Accurate mapping and timely monitoring of urban redevelopment are pivotal for urban studies and decisionmakers to foster sustainable urban development.Traditional mapping methods heavily depend on field surveys and subjective questionnaires,yielding less objective,reliable,and timely data.Recent advancements in Geographic Information Systems(GIS)and remote-sensing technologies have improved the identification and mapping of urban redevelopment through quantitative analysis using satellite-based observations.Nonetheless,challenges persist,particularly concerning accuracy and significant temporal delays.This study introduces a novel approach to modeling urban redevelopment,leveraging machine learning algorithms and remote-sensing data.This methodology can facilitate the accurate and timely identification of urban redevelopment activities.The study’s machine learning model can analyze time-series remote-sensing data to identify spatio-temporal and spectral patterns related to urban redevelopment.The model is thoroughly evaluated,and the results indicate that it can accurately capture the time-series patterns of urban redevelopment.This research’s findings are useful for evaluating urban demographic and economic changes,informing policymaking and urban planning,and contributing to sustainable urban development.The model can also serve as a foundation for future research on early-stage urban redevelopment detection and evaluation of the causes and impacts of urban redevelopment.

Missing Value Imputation for Radar-Derived Time-Series Tracks of Aerial Targets Based on Improved Self-Attention-Based Network

The frequent missing values in radar-derived time-series tracks of aerial targets(RTT-AT)lead to significant challenges in subsequent data-driven tasks.However,the majority of imputation research focuses on random missing(RM)that differs significantly from common missing patterns of RTT-AT.The method for solving the RM may experience performance degradation or failure when applied to RTT-AT imputation.Conventional autoregressive deep learning methods are prone to error accumulation and long-term dependency loss.In this paper,a non-autoregressive imputation model that addresses the issue of missing value imputation for two common missing patterns in RTT-AT is proposed.Our model consists of two probabilistic sparse diagonal masking self-attention(PSDMSA)units and a weight fusion unit.It learns missing values by combining the representations outputted by the two units,aiming to minimize the difference between the missing values and their actual values.The PSDMSA units effectively capture temporal dependencies and attribute correlations between time steps,improving imputation quality.The weight fusion unit automatically updates the weights of the output representations from the two units to obtain a more accurate final representation.The experimental results indicate that,despite varying missing rates in the two missing patterns,our model consistently outperforms other methods in imputation performance and exhibits a low frequency of deviations in estimates for specific missing entries.Compared to the state-of-the-art autoregressive deep learning imputation model Bidirectional Recurrent Imputation for Time Series(BRITS),our proposed model reduces mean absolute error(MAE)by 31%~50%.Additionally,the model attains a training speed that is 4 to 8 times faster when compared to both BRITS and a standard Transformer model when trained on the same dataset.Finally,the findings from the ablation experiments demonstrate that the PSDMSA,the weight fusion unit,cascade network design,and imputation loss enhance imputation performance and confirm the efficacy of our design.

Meteorological factors, ambient air pollution, and daily hospital admissions for depressive disorder in Harbin: A time-series study

BACKGROUND The literature has discussed the relationship between environmental factors and depressive disorders;however,the results are inconsistent in different studies and regions,as are the interaction effects between environmental factors.We hypo-thesized that meteorological factors and ambient air pollution individually affect and interact to affect depressive disorder morbidity.AIM To investigate the effects of meteorological factors and air pollution on depressive disorders,including their lagged effects and interactions.METHODS The samples were obtained from a class 3 hospital in Harbin,China.Daily hos-pital admission data for depressive disorders from January 1,2015 to December 31,2022 were obtained.Meteorological and air pollution data were also collected during the same period.Generalized additive models with quasi-Poisson regre-ssion were used for time-series modeling to measure the non-linear and delayed effects of environmental factors.We further incorporated each pair of environ-mental factors into a bivariate response surface model to examine the interaction effects on hospital admissions for depressive disorders.RESULTS Data for 2922 d were included in the study,with no missing values.The total number of depressive admissions was 83905.Medium to high correlations existed between environmental factors.Air temperature(AT)and wind speed(WS)significantly affected the number of admissions for depression.An extremely low temperature(-29.0℃)at lag 0 caused a 53%[relative risk(RR)=1.53,95%confidence interval(CI):1.23-1.89]increase in daily hospital admissions relative to the median temperature.Extremely low WSs(0.4 m/s)at lag 7 increased the number of admissions by 58%(RR=1.58,95%CI:1.07-2.31).In contrast,atmospheric pressure and relative humidity had smaller effects.Among the six air pollutants considered in the time-series model,nitrogen dioxide(NO_(2))was the only pollutant that showed significant effects over non-cumulative,cumulative,immediate,and lagged conditions.The cumulative effect of NO_(2) at lag 7 was 0.47%(RR=1.0047,95%CI:1.0024-1.0071).Interaction effects were found between AT and the five air pollutants,atmospheric temperature and the four air pollutants,WS and sulfur dioxide.CONCLUSION Meteorological factors and the air pollutant NO_(2) affect daily hospital admissions for depressive disorders,and interactions exist between meteorological factors and ambient air pollution.

Review of the SBAS InSAR Time-series algorithms, applications, and challenges

In the past 30 years,the small baseline subset(SBAS)InSAR time-series technique has emerged as an essential tool for measuring slow surface displacement and estimating geophysical parameters.Because of its ability to monitor large-scale deformation with millimeter accuracy,the SBAS method has been widely used in various geodetic fields,such as ground subsidence,landslides,and seismic activity.The obtained long-term time-series cumulative deformation is vital for studying the deformation mecha-nism.This article reviews the algorithms,applications,and challenges of the SBAS method.First,we recall the fundamental principle and analyze the shortcomings of the traditional SBAS algorithm,which provides a basic framework for the following improved time series methods.Second,we classify the current improved SBAS techniques from different perspectives:solving the ill-posed equation,increasing the density of high-coherence points,improving the accuracy of monitoring deformation and measuring the multi-dimensional deformation.Third,we summarize the application of the SBAS method in monitoring ground subsidence,permafrost degradation,glacier movement,volcanic activity,landslides,and seismic activity.Finally,we discuss the difficulties faced by the SBAS method and explore its future development direction.

Graph Construction Method for GNN-Based Multivariate Time-Series Forecasting

Multivariate time-series forecasting(MTSF)plays an important role in diverse real-world applications.To achieve better accuracy in MTSF,time-series patterns in each variable and interrelationship patterns between variables should be considered together.Recently,graph neural networks(GNNs)has gained much attention as they can learn both patterns using a graph.For accurate forecasting through GNN,a well-defined graph is required.However,existing GNNs have limitations in reflecting the spectral similarity and time delay between nodes,and consider all nodes with the same weight when constructing graph.In this paper,we propose a novel graph construction method that solves aforementioned limitations.We first calculate the Fourier transform-based spectral similarity and then update this similarity to reflect the time delay.Then,we weight each node according to the number of edge connections to get the final graph and utilize it to train the GNN model.Through experiments on various datasets,we demonstrated that the proposed method enhanced the performance of GNN-based MTSF models,and the proposed forecasting model achieve of up to 18.1%predictive performance improvement over the state-of-the-art model.

Sentinel-1 In SAR observations and time-series analysis of co-and postseismic deformation mechanisms of the 2021 Mw 5.8 Bandar Ganaveh Earthquake,Southern Iran

In the past two decades,because of the significant increase in the availability of differential interferometry from synthetic aperture radar and GPS data,spaceborne geodesy has been widely employed to determine the co-seismic displacement field of earthquakes.On April 18,2021,a moderate earthquake(Mw 5.8)occurred east of Bandar Ganaveh,southern Iran,followed by intensive seismic activity and aftershocks of various magnitudes.We use two-pass D-InSAR and Small Baseline Inversion techniques via the LiCSBAS suite to study the coseismic displacement and monitor the four-month post-seismic deformation of the Bandar Ganaveh earthquake,as well as constrain the fault geometry of the co-seismic faulting mechanism during the seismic sequence.Analyses show that the co-and postseismic deformation are distributed in relatively shallow depths along with an NW-SE striking and NE dipping complex reverse/thrust fault branches of the Zagros Mountain Front Fault,complying with the main trend of the Zagros structures.The average cumulative displacements were obtained from-137.5 to+113.3 mm/yr in the SW and NE blocks of the Mountain Front Fault,respectively.The received maximum uplift amount is approximately consistent with the overall orogen-normal shortening component of the Arabian-Eurasian convergence in the Zagros region.No surface ruptures were associated with the seismic source;therefore,we propose a shallow blind thrust/reverse fault(depth~10 km)connected to the deeper basal decollement fault within a complex tectonic zone,emphasizing the thin-skinned tectonics.

Generating Time-Series Data Using Generative Adversarial Networks for Mobility Demand Prediction

The increasing penetration rate of electric kickboard vehicles has been popularized and promoted primarily because of its clean and efficient features.Electric kickboards are gradually growing in popularity in tourist and education-centric localities.In the upcoming arrival of electric kickboard vehicles,deploying a customer rental service is essential.Due to its freefloating nature,the shared electric kickboard is a common and practical means of transportation.Relocation plans for shared electric kickboards are required to increase the quality of service,and forecasting demand for their use in a specific region is crucial.Predicting demand accurately with small data is troublesome.Extensive data is necessary for training machine learning algorithms for effective prediction.Data generation is a method for expanding the amount of data that will be further accessible for training.In this work,we proposed a model that takes time-series customers’electric kickboard demand data as input,pre-processes it,and generates synthetic data according to the original data distribution using generative adversarial networks(GAN).The electric kickboard mobility demand prediction error was reduced when we combined synthetic data with the original data.We proposed Tabular-GAN-Modified-WGAN-GP for generating synthetic data for better prediction results.We modified The Wasserstein GAN-gradient penalty(GP)with the RMSprop optimizer and then employed Spectral Normalization(SN)to improve training stability and faster convergence.Finally,we applied a regression-based blending ensemble technique that can help us to improve performance of demand prediction.We used various evaluation criteria and visual representations to compare our proposed model’s performance.Synthetic data generated by our suggested GAN model is also evaluated.The TGAN-Modified-WGAN-GP model mitigates the overfitting and mode collapse problem,and it also converges faster than previous GAN models for synthetic data creation.The presented model’s performance is compared to existing ensemble and baseline models.The experimental findings imply that combining synthetic and actual data can significantly reduce prediction error rates in the mean absolute percentage error(MAPE)of 4.476 and increase prediction accuracy.

Integrative bioinformatics and in vitro exploration of EVI2A expression:unraveling its immunological and prognostic implications in kidney renal clear cell carcinoma

EVI2A has emerged as a significant biomarker in various diseases;however,its biological role and mechanism in kidney renal clear cell carcinoma(KIRC)remains unexplored.We used TCGA and GEO databases to analyze EVI2A gene expression comprehensively and performed pan-cancer assessments.Clinical relevance was evaluated through Kaplan-Meier analysis and ROC curves.The gene’s immune relevance was explored through analyses of the tumor microenvironment(TME),Tumor Immune Single-cell Hub(TISCH),immune checkpoints,and immunotherapy sensitivity.Our results indicate that EVI2A expression is upregulated in KIRC,showing correlations with tumor grade and T/N/M stage.EVI2A demonstrates high diagnostic accuracy(AUC=0.906)and predicts poor overall and progression-free survival in KIRC patients.Furthermore,EVI2A expression exhibits significant associations with immunity,including TME scores and specific immune cell types such as Tfh cells,CD4 memory T cells,and CD8+T cells.Elevated EVI2A expression suggests increased sensitivity to PD-1/CTLA-4 and tyrosine kinase inhibitors.In vitro assays confirmed the impact of EVI2A on KIRC behavior,with its knockdown resulting in reduced cell proliferation and migration.In conclusion,our comprehensive analysis identifies EVI2A as a promising biomarker and a novel therapeutic target for intervening in KIRC.These findings hold significant implications for further research and potential clinical applications.

面向新能源时代的汽车用钢EVI工程技术发展

由于能源和环境问题,全球新能源汽车产业得到快速发展,而高强钢由于其高强度、可加工性、高性价比优势,其在新能源汽车上的集成应用成为业界的焦点领域。基于供应商的早期介入(EVI)理念,上下游产业链先后开展了多款钢制轻量化车身项目,如浦项PBC-EV、蒂森In-car、安米S-in motion、宝钢BCB系列和马钢MCEV等,主要围绕车身及电池包等主要部件。未来随着汽车产业电动化、节能减排政策的实施以及关键技术的成熟,先进高强钢在汽车轻量化领域将发挥更大作用。

基于NDVI和EVI不同植被指数表征的粤港澳大湾区植被空间格局驱动因子影响力比较分析

为研究归一化植被指数(normalized difference vegetation index,NDVI)和增强植被指数(enhanced vegetation index,EVI)的表征差异是否会造成有关植被研究的结果差异,分别在采用2005—2020年MODISNDVI、MODIS-EVI 2种不同遥感植被指数表征粤港澳大湾区植被空间特征的基础上,以同时期17个自然因子和人为因子作为驱动因子,通过地理探测器模型方法,计算各驱动因子对基于NDVI和EVI的植被空间特征的影响力。结果显示,虽然粤港澳大湾区南亚热带-热带植被在NDVI和EVI的表征下,其结果存在差异,但在不同植被指数下,通过地理探测器模型方法计算各驱动因子影响力量化及排序结果基本一致,未受不同植被指数表征差异的影响。在驱动因子中,土地利用类型、高程均是最主要驱动因子,对植被空间分布影响力均超过50%。因子之间均表现出双因子增强作用。土地利用类型协同人口分布因子对NDVI表征下的植被空间分布影响力最强;高程协同人口分布因子对EVI表征下的植被空间分布影响力最强。研究结果表明,虽然NDVI、EVI在表征植被覆盖特征方面存在差异,但是基于不同植被指数计算的驱动因子定量分析结果趋于一致。

Temperature and Daily Mortality in Shanghai:A Time-series Study

To investigate the association between temperature and daily mortality in Shanghai from June 1, 2000 to December 31, 2001. Methods Time-series approach was used to estimate the effect of temperature on daily total and cause-specific mortality. We fitted generalized additive Poisson regression using non-parametric smooth functions to control for long-term time trend, season and other variables. We also controlled for day of the week. Results A gently sloping V-like relationship between total mortality and temperature was found, with an optimum temperature (e.g. temperature with lowest mortality risk) value of 26.7癈 in Shanghai. For temperatures above the optimum value, total mortality increased by 0.73% for each degree Celsius increase; while for temperature below the optimum value, total mortality decreased by 1.21% for each degree Celsius increase. Conclusions Our findings indicate that temperature has an effect on daily mortality in Shanghai, and the time-series approach is a useful tool for studying the temperature-mortality association.

Mapping winter wheat using phenological feature of peak before winter on the North China Plain based on time-series MODIS data

By employing the unique phenological feature of winter wheat extracted from peak before winter （PBW） and the advantages of moderate resolution imaging spectroradiometer （MODIS） data with high temporal resolution and intermediate spatial resolution, a remote sensing-based model for mapping winter wheat on the North China Plain was built through integration with Landsat images and land-use data. First, a phenological window, PBW was drawn from time-series MODIS data. Next, feature extraction was performed for the PBW to reduce feature dimension and enhance its information. Finally, a regression model was built to model the relationship of the phenological feature and the sample data. The amount of information of the PBW was evaluated and compared with that of the main peak （MP）. The relative precision of the mapping reached up to 92% in comparison to the Landsat sample data, and ranged between 87 and 96% in comparison to the statistical data. These results were sufficient to satisfy the accuracy requirements for winter wheat mapping at a large scale. Moreover, the proposed method has the ability to obtain the distribution information for winter wheat in an earlier period than previous studies. This study could throw light on the monitoring of winter wheat in China by using unique phenological feature of winter wheat.

Clustering Structure Analysis in Time-Series Data With Density-Based Clusterability Measure

Clustering is used to gain an intuition of the struc tures in the data.Most of the current clustering algorithms pro duce a clustering structure even on data that do not possess such structure.In these cases,the algorithms force a structure in the data instead of discovering one.To avoid false structures in the relations of data,a novel clusterability assessment method called density-based clusterability measure is proposed in this paper.I measures the prominence of clustering structure in the data to evaluate whether a cluster analysis could produce a meaningfu insight to the relationships in the data.This is especially useful in time-series data since visualizing the structure in time-series data is hard.The performance of the clusterability measure is evalu ated against several synthetic data sets and time-series data sets which illustrate that the density-based clusterability measure can successfully indicate clustering structure of time-series data.

Wavelet matrix transform for time-series similarity measurement

A time-series similarity measurement method based on wavelet and matrix transform was proposed,and its anti-noise ability,sensitivity and accuracy were discussed. The time-series sequences were compressed into wavelet subspace,and sample feature vector and orthogonal basics of sample time-series sequences were obtained by K-L transform. Then the inner product transform was carried out to project analyzed time-series sequence into orthogonal basics to gain analyzed feature vectors. The similarity was calculated between sample feature vector and analyzed feature vector by the Euclid distance. Taking fault wave of power electronic devices for example,the experimental results show that the proposed method has low dimension of feature vector,the anti-noise ability of proposed method is 30 times as large as that of plain wavelet method,the sensitivity of proposed method is 1/3 as large as that of plain wavelet method,and the accuracy of proposed method is higher than that of the wavelet singular value decomposition method. The proposed method can be applied in similarity matching and indexing for lager time series databases.

Time-series gas prediction model using LS-SVR within a Bayesian framework

The traditional least squares support vector regression（LS-SVR）model,using cross validation to determine the regularization parameter and kernel parameter,is time-consuming.We propose a Bayesian evidence framework to infer the LS-SVR model parameters.Three levels Bayesian inferences are used to determine the model parameters,regularization hyper-parameters and tune the nuclear parameters by model comparison.On this basis,we established Bayesian LS-SVR time-series gas forecasting models and provide steps for the algorithm.The gas outburst data of a Hebi 10th mine working face is used to validate the model.The optimal embedding dimension and delay time of the time series were obtained by the smallest differential entropy method.Finally,within a MATLAB7.1 environment,we used actual coal gas data to compare the traditional LS-SVR and the Bayesian LS-SVR with LS-SVMlab1.5 Toolbox simulation.The results show that the Bayesian framework of an LS-SVR significantly improves the speed and accuracy of the forecast.

Spatio-temporal changes of underground coal fires during 2008-2016 in Khanh Hoa coal field(North-east of Viet Nam) using Landsat time-series data

Underground coal fires are one of the most common and serious geohazards in most coal producing countries in the world. Monitoring their spatio-temporal changes plays an important role in controlling and preventing the effects of coal fires, and their environmental impact. In this study, the spatio-temporal changes of underground coal fires in Khanh Hoa coal field(North-East of Viet Nam) were analyzed using Landsat time-series data during the 2008-2016 period. Based on land surface temperatures retrieved from Landsat thermal data, underground coal fires related to thermal anomalies were identified using the MEDIAN+1.5×IQR(IQR: Interquartile range) threshold technique. The locations of underground coal fires were validated using a coal fire map produced by the field survey data and cross-validated using the daytime ASTER thermal infrared imagery. Based on the fires extracted from seven Landsat thermal imageries, the spatiotemporal changes of underground coal fire areas were analyzed. The results showed that the thermalanomalous zones have been correlated with known coal fires. Cross-validation of coal fires using ASTER TIR data showed a high consistency of 79.3%. The largest coal fire area of 184.6 hectares was detected in 2010, followed by 2014(181.1 hectares) and 2016(178.5 hectares). The smaller coal fire areas were extracted with areas of 133.6 and 152.5 hectares in 2011 and 2009 respectively. Underground coal fires were mainly detected in the northern and southern part, and tend to spread to north-west of the coal field.

Time-series gene expression prof iles in AGS cells stimulated with Helicobacter pylori

AIM: To extend the knowledge of the dynamic interaction between Helicobacter pylori (H. pylori) and host mucosa. METHODS: A time-series cDNA microarray was performed in order to detect the temporal gene expression prof iles of human gastric epithelial adenocarcinoma cells infected with H. pylori. Six time points were selected to observe the changes in the model. A differential expression prof ile at each time point was obtained by comparing the microarray signal value with that of 0 h. Real-time polymerase chain reaction was subsequently performed to evaluate the data quality. RESULTS: We found a diversity of gene expression patterns at different time points and identifi ed a group of genes whose expression levels were significantly correlated with several important immune response and tumor related pathways. CONCLUSION: Early infection may trigger some important pathways and may impact the outcome of the infection.

Monitoring of winter wheat distribution and phenological phases based on MODIS time-series： A case study in the Yellow River Delta, China

Accurate winter wheat identification and phenology extraction are essential for field management and agricultural policy making. Here, we present mechanisms of winter wheat discrimination and phenological detection in the Yellow River Delta（YRD） region using moderate resolution imaging spectroradiometer（MODIS） time-series data. The normalized difference vegetation index（NDVI） was obtained by calculating the surface reflectance in red and infrared. We used the Savitzky-Golay filter to smooth time series NDVI curves. We adopted a two-step classification to identify winter wheat. The first step was designed to mask out non-vegetation classes, and the second step aimed to identify winter wheat from other vegetation based on its phenological features. We used the double Gaussian model and the maximum curvature method to extract phenology. Due to the characteristics of the time-series profiles for winter wheat, a double Gaussian function method was selected to fit the temporal profile. A maximum curvature method was performed to extract phenological phases. Phenological phases such as the green-up, heading and harvesting phases were detected when the NDVI curvature exhibited local maximum values. The extracted phenological dates then were validated with records of the ground observations. The spatial patterns of phenological phases were investigated. This study concluded that, for winter wheat, the accuracy of classification is 87.07%, and the accuracy of planting acreage is 90.09%. The phenological result was comparable to the ground observation at the municipal level. The average green-up date for the whole region occurred on March 5, the average heading date occurred on May 9, and the average harvesting date occurred on June 5. The spatial distribution of the phenology for winter wheat showed a significant gradual delay from the southwest to the northeast. This study demonstrates the effectiveness of our proposed method for winter wheat classification and phenology detection.

TIME-SERIES MODELI NG AND FAULT FORECAST STUDY ON SPECTRAL ANALYSIS OF LUBRICATING OIL

The application of ti me-series modeling and forecasting method to the spectral analysis for lubricat ing oil of mechanical equipment is discussed. The AR model is used to perform a time-series modeling and forecasting analysis for the spectral analysis data co llected from aero-engines. In the oil condition monitoring field of mechanical equipment, the use of the method of time-series analysis has rarely been report ed. As indicated in the satisfactory example, a practical method for condition m onitoring and fault forecasting of mechanical equipment has been achieved.