Hierarchical multihead self-attention for time-series-based fault diagnosis

Fault diagnosis is important for maintaining the safety and effectiveness of chemical process.Considering the multivariate,nonlinear,and dynamic characteristic of chemical process,many time-series-based data-driven fault diagnosis methods have been developed in recent years.However,the existing methods have the problem of long-term dependency and are difficult to train due to the sequential way of training.To overcome these problems,a novel fault diagnosis method based on time-series and the hierarchical multihead self-attention(HMSAN)is proposed for chemical process.First,a sliding window strategy is adopted to construct the normalized time-series dataset.Second,the HMSAN is developed to extract the time-relevant features from the time-series process data.It improves the basic self-attention model in both width and depth.With the multihead structure,the HMSAN can pay attention to different aspects of the complicated chemical process and obtain the global dynamic features.However,the multiple heads in parallel lead to redundant information,which cannot improve the diagnosis performance.With the hierarchical structure,the redundant information is reduced and the deep local time-related features are further extracted.Besides,a novel many-to-one training strategy is introduced for HMSAN to simplify the training procedure and capture the long-term dependency.Finally,the effectiveness of the proposed method is demonstrated by two chemical cases.The experimental results show that the proposed method achieves a great performance on time-series industrial data and outperforms the state-of-the-art approaches.

Modeling urban redevelopment:A novel approach using time-series remote sensing data and machine learning

Accurate mapping and timely monitoring of urban redevelopment are pivotal for urban studies and decisionmakers to foster sustainable urban development.Traditional mapping methods heavily depend on field surveys and subjective questionnaires,yielding less objective,reliable,and timely data.Recent advancements in Geographic Information Systems(GIS)and remote-sensing technologies have improved the identification and mapping of urban redevelopment through quantitative analysis using satellite-based observations.Nonetheless,challenges persist,particularly concerning accuracy and significant temporal delays.This study introduces a novel approach to modeling urban redevelopment,leveraging machine learning algorithms and remote-sensing data.This methodology can facilitate the accurate and timely identification of urban redevelopment activities.The study’s machine learning model can analyze time-series remote-sensing data to identify spatio-temporal and spectral patterns related to urban redevelopment.The model is thoroughly evaluated,and the results indicate that it can accurately capture the time-series patterns of urban redevelopment.This research’s findings are useful for evaluating urban demographic and economic changes,informing policymaking and urban planning,and contributing to sustainable urban development.The model can also serve as a foundation for future research on early-stage urban redevelopment detection and evaluation of the causes and impacts of urban redevelopment.

Missing Value Imputation for Radar-Derived Time-Series Tracks of Aerial Targets Based on Improved Self-Attention-Based Network

The frequent missing values in radar-derived time-series tracks of aerial targets(RTT-AT)lead to significant challenges in subsequent data-driven tasks.However,the majority of imputation research focuses on random missing(RM)that differs significantly from common missing patterns of RTT-AT.The method for solving the RM may experience performance degradation or failure when applied to RTT-AT imputation.Conventional autoregressive deep learning methods are prone to error accumulation and long-term dependency loss.In this paper,a non-autoregressive imputation model that addresses the issue of missing value imputation for two common missing patterns in RTT-AT is proposed.Our model consists of two probabilistic sparse diagonal masking self-attention(PSDMSA)units and a weight fusion unit.It learns missing values by combining the representations outputted by the two units,aiming to minimize the difference between the missing values and their actual values.The PSDMSA units effectively capture temporal dependencies and attribute correlations between time steps,improving imputation quality.The weight fusion unit automatically updates the weights of the output representations from the two units to obtain a more accurate final representation.The experimental results indicate that,despite varying missing rates in the two missing patterns,our model consistently outperforms other methods in imputation performance and exhibits a low frequency of deviations in estimates for specific missing entries.Compared to the state-of-the-art autoregressive deep learning imputation model Bidirectional Recurrent Imputation for Time Series(BRITS),our proposed model reduces mean absolute error(MAE)by 31%~50%.Additionally,the model attains a training speed that is 4 to 8 times faster when compared to both BRITS and a standard Transformer model when trained on the same dataset.Finally,the findings from the ablation experiments demonstrate that the PSDMSA,the weight fusion unit,cascade network design,and imputation loss enhance imputation performance and confirm the efficacy of our design.

Meteorological factors, ambient air pollution, and daily hospital admissions for depressive disorder in Harbin: A time-series study

BACKGROUND The literature has discussed the relationship between environmental factors and depressive disorders;however,the results are inconsistent in different studies and regions,as are the interaction effects between environmental factors.We hypo-thesized that meteorological factors and ambient air pollution individually affect and interact to affect depressive disorder morbidity.AIM To investigate the effects of meteorological factors and air pollution on depressive disorders,including their lagged effects and interactions.METHODS The samples were obtained from a class 3 hospital in Harbin,China.Daily hos-pital admission data for depressive disorders from January 1,2015 to December 31,2022 were obtained.Meteorological and air pollution data were also collected during the same period.Generalized additive models with quasi-Poisson regre-ssion were used for time-series modeling to measure the non-linear and delayed effects of environmental factors.We further incorporated each pair of environ-mental factors into a bivariate response surface model to examine the interaction effects on hospital admissions for depressive disorders.RESULTS Data for 2922 d were included in the study,with no missing values.The total number of depressive admissions was 83905.Medium to high correlations existed between environmental factors.Air temperature(AT)and wind speed(WS)significantly affected the number of admissions for depression.An extremely low temperature(-29.0℃)at lag 0 caused a 53%[relative risk(RR)=1.53,95%confidence interval(CI):1.23-1.89]increase in daily hospital admissions relative to the median temperature.Extremely low WSs(0.4 m/s)at lag 7 increased the number of admissions by 58%(RR=1.58,95%CI:1.07-2.31).In contrast,atmospheric pressure and relative humidity had smaller effects.Among the six air pollutants considered in the time-series model,nitrogen dioxide(NO_(2))was the only pollutant that showed significant effects over non-cumulative,cumulative,immediate,and lagged conditions.The cumulative effect of NO_(2) at lag 7 was 0.47%(RR=1.0047,95%CI:1.0024-1.0071).Interaction effects were found between AT and the five air pollutants,atmospheric temperature and the four air pollutants,WS and sulfur dioxide.CONCLUSION Meteorological factors and the air pollutant NO_(2) affect daily hospital admissions for depressive disorders,and interactions exist between meteorological factors and ambient air pollution.

Dynamics and vibration reduction performance of asymmetric tristable nonlinear energy sink

With its complex nonlinear dynamic behavior,the tristable system has shown excellent performance in areas such as energy harvesting and vibration suppression,and has attracted a lot of attention.In this paper,an asymmetric tristable design is proposed to improve the vibration suppression efficiency of nonlinear energy sinks(NESs)for the first time.The proposed asymmetric tristable NES(ATNES)is composed of a pair of oblique springs and a vertical spring.Then,the three stable states,symmetric and asymmetric,can be achieved by the adjustment of the distance and stiffness asymmetry of the oblique springs.The governing equations of a linear oscillator(LO)coupled with the ATNES are derived.The approximate analytical solution to the coupled system is obtained by the harmonic balance method(HBM)and verified numerically.The vibration suppression efficiency of three types of ATNES is compared.The results show that the asymmetric design can improve the efficiency of vibration reduction through comparing the chaotic motion of the NES oscillator between asymmetric steady states.In addition,compared with the symmetrical tristable NES(TNES),the ATNES can effectively control smaller structural vibrations.In other words,the ATNES can effectively solve the threshold problem of TNES failure to weak excitation.Therefore,this paper reveals the vibration reduction mechanism of the ATNES,and provides a pathway to expand the effective excitation amplitude range of the NES.

Snap-through behaviors and nonlinear vibrations of a bistable composite laminated cantilever shell:an experimental and numerical study

The snap-through behaviors and nonlinear vibrations are investigated for a bistable composite laminated cantilever shell subjected to transversal foundation excitation based on experimental and theoretical approaches.An improved experimental specimen is designed in order to satisfy the cantilever support boundary condition,which is composed of an asymmetric region and a symmetric region.The symmetric region of the experimental specimen is entirely clamped,which is rigidly connected to an electromagnetic shaker,while the asymmetric region remains free of constraint.Different motion paths are realized for the bistable cantilever shell by changing the input signal levels of the electromagnetic shaker,and the displacement responses of the shell are collected by the laser displacement sensors.The numerical simulation is conducted based on the established theoretical model of the bistable composite laminated cantilever shell,and an off-axis three-dimensional dynamic snap-through domain is obtained.The numerical solutions are in good agreement with the experimental results.The nonlinear stiffness characteristics,dynamic snap-through domain,and chaos and bifurcation behaviors of the shell are quantitatively analyzed.Due to the asymmetry of the boundary condition and the shell,the upper stable-state of the shell exhibits an obvious soft spring stiffness characteristic,and the lower stable-state shows a linear stiffness characteristic of the shell.

Graph Construction Method for GNN-Based Multivariate Time-Series Forecasting

Multivariate time-series forecasting(MTSF)plays an important role in diverse real-world applications.To achieve better accuracy in MTSF,time-series patterns in each variable and interrelationship patterns between variables should be considered together.Recently,graph neural networks(GNNs)has gained much attention as they can learn both patterns using a graph.For accurate forecasting through GNN,a well-defined graph is required.However,existing GNNs have limitations in reflecting the spectral similarity and time delay between nodes,and consider all nodes with the same weight when constructing graph.In this paper,we propose a novel graph construction method that solves aforementioned limitations.We first calculate the Fourier transform-based spectral similarity and then update this similarity to reflect the time delay.Then,we weight each node according to the number of edge connections to get the final graph and utilize it to train the GNN model.Through experiments on various datasets,we demonstrated that the proposed method enhanced the performance of GNN-based MTSF models,and the proposed forecasting model achieve of up to 18.1%predictive performance improvement over the state-of-the-art model.

Sentinel-1 In SAR observations and time-series analysis of co-and postseismic deformation mechanisms of the 2021 Mw 5.8 Bandar Ganaveh Earthquake,Southern Iran

In the past two decades,because of the significant increase in the availability of differential interferometry from synthetic aperture radar and GPS data,spaceborne geodesy has been widely employed to determine the co-seismic displacement field of earthquakes.On April 18,2021,a moderate earthquake(Mw 5.8)occurred east of Bandar Ganaveh,southern Iran,followed by intensive seismic activity and aftershocks of various magnitudes.We use two-pass D-InSAR and Small Baseline Inversion techniques via the LiCSBAS suite to study the coseismic displacement and monitor the four-month post-seismic deformation of the Bandar Ganaveh earthquake,as well as constrain the fault geometry of the co-seismic faulting mechanism during the seismic sequence.Analyses show that the co-and postseismic deformation are distributed in relatively shallow depths along with an NW-SE striking and NE dipping complex reverse/thrust fault branches of the Zagros Mountain Front Fault,complying with the main trend of the Zagros structures.The average cumulative displacements were obtained from-137.5 to+113.3 mm/yr in the SW and NE blocks of the Mountain Front Fault,respectively.The received maximum uplift amount is approximately consistent with the overall orogen-normal shortening component of the Arabian-Eurasian convergence in the Zagros region.No surface ruptures were associated with the seismic source;therefore,we propose a shallow blind thrust/reverse fault(depth~10 km)connected to the deeper basal decollement fault within a complex tectonic zone,emphasizing the thin-skinned tectonics.

Generating Time-Series Data Using Generative Adversarial Networks for Mobility Demand Prediction

The increasing penetration rate of electric kickboard vehicles has been popularized and promoted primarily because of its clean and efficient features.Electric kickboards are gradually growing in popularity in tourist and education-centric localities.In the upcoming arrival of electric kickboard vehicles,deploying a customer rental service is essential.Due to its freefloating nature,the shared electric kickboard is a common and practical means of transportation.Relocation plans for shared electric kickboards are required to increase the quality of service,and forecasting demand for their use in a specific region is crucial.Predicting demand accurately with small data is troublesome.Extensive data is necessary for training machine learning algorithms for effective prediction.Data generation is a method for expanding the amount of data that will be further accessible for training.In this work,we proposed a model that takes time-series customers’electric kickboard demand data as input,pre-processes it,and generates synthetic data according to the original data distribution using generative adversarial networks(GAN).The electric kickboard mobility demand prediction error was reduced when we combined synthetic data with the original data.We proposed Tabular-GAN-Modified-WGAN-GP for generating synthetic data for better prediction results.We modified The Wasserstein GAN-gradient penalty(GP)with the RMSprop optimizer and then employed Spectral Normalization(SN)to improve training stability and faster convergence.Finally,we applied a regression-based blending ensemble technique that can help us to improve performance of demand prediction.We used various evaluation criteria and visual representations to compare our proposed model’s performance.Synthetic data generated by our suggested GAN model is also evaluated.The TGAN-Modified-WGAN-GP model mitigates the overfitting and mode collapse problem,and it also converges faster than previous GAN models for synthetic data creation.The presented model’s performance is compared to existing ensemble and baseline models.The experimental findings imply that combining synthetic and actual data can significantly reduce prediction error rates in the mean absolute percentage error(MAPE)of 4.476 and increase prediction accuracy.

Prescribed Performance Tracking Control of Time-Delay Nonlinear Systems With Output Constraints

The problem of prescribed performance tracking control for unknown time-delay nonlinear systems subject to output constraints is dealt with in this paper. In contrast with related works, only the most fundamental requirements, i.e., boundedness and the local Lipschitz condition, are assumed for the allowable time delays. Moreover, we focus on the case where the reference is unknown beforehand, which renders the standard prescribed performance control designs under output constraints infeasible. To conquer these challenges, a novel robust prescribed performance control approach is put forward in this paper.Herein, a reverse tuning function is skillfully constructed and automatically generates a performance envelop for the tracking error. In addition, a unified performance analysis framework based on proof by contradiction and the barrier function is established to reveal the inherent robustness of the control system against the time delays. It turns out that the system output tracks the reference with a preassigned settling time and good accuracy,without constraint violations. A comparative simulation on a two-stage chemical reactor is carried out to illustrate the above theoretical findings.

Underwater four-quadrant dual-beam circumferential scanning laser fuze using nonlinear adaptive backscatter filter based on pauseable SAF-LMS algorithm

The phenomenon of a target echo peak overlapping with the backscattered echo peak significantly undermines the detection range and precision of underwater laser fuzes.To overcome this issue,we propose a four-quadrant dual-beam circumferential scanning laser fuze to distinguish various interference signals and provide more real-time data for the backscatter filtering algorithm.This enhances the algorithm loading capability of the fuze.In order to address the problem of insufficient filtering capacity in existing linear backscatter filtering algorithms,we develop a nonlinear backscattering adaptive filter based on the spline adaptive filter least mean square(SAF-LMS)algorithm.We also designed an algorithm pause module to retain the original trend of the target echo peak,improving the time discrimination accuracy and anti-interference capability of the fuze.Finally,experiments are conducted with varying signal-to-noise ratios of the original underwater target echo signals.The experimental results show that the average signal-to-noise ratio before and after filtering can be improved by more than31 d B,with an increase of up to 76%in extreme detection distance.

Resonantly enhanced second-and third-harmonic generation in dielectric nonlinear metasurfaces

Nonlinear dielectric metasurfaces provide a promising approach to control and manipulate frequency conversion optical processes at the nanoscale,thus facilitating both advances in fundamental research and the development of new practical applications in photonics,lasing,and sensing.Here,we employ symmetry-broken metasurfaces made of centrosymmetric amorphous silicon for resonantly enhanced second-and third-order nonlinear optical response.Exploiting the rich physics of optical quasi-bound states in the continuum and guided mode resonances,we comprehensively study through rigorous numerical calculations the relative contribution of surface and bulk effects to second-harmonic generation(SHG)and the bulk contribution to third-harmonic generation(THG) from the meta-atoms.Next,we experimentally achieve optical resonances with high quality factors,which greatly boosts light-matter interaction,resulting in about 550 times SHG enhancement and nearly 5000-fold increase of THG.A good agreement between theoretical predictions and experimental measurements is observed.To gain deeper insights into the physics of the investigated nonlinear optical processes,we further numerically study the relation between nonlinear emission and the structural asymmetry of the metasurface and reveal that the generated harmonic signals arising from linear sharp resonances are highly dependent on the asymmetry of the meta-atoms.Our work suggests a fruitful strategy to enhance the harmonic generation and effectively control different orders of harmonics in all-dielectric metasurfaces,enabling the development of efficient active photonic nanodevices.

Geometric Programming for Nonlinear Satellite Buffer Networks With Time Delays under L_(1)-Gain Performance

Dear Editor,This letter concerns the parameter tuning problem for nonlinear satellite buffer networks with communication delays, aiming to optimize their stability properties under L_(1)-gain. We first model the satellite buffer networks by a nonlinear time-delay positive system and propose a novel characterization under which the nonlinear system is asymptotically stable with a prescribed L_(1)-induced performance.

Nonlinear phenomena in vibrations of embedded carbon nanotubes conveying viscous fluid

Various nonlinear phenomena such as bifurcations and chaos in the responses of carbon nanotubes(CNTs)are recognized as being major contributors to the inaccuracy and instability of nanoscale mechanical systems.Therefore,the main purpose of this paper is to predict the nonlinear dynamic behavior of a CNT conveying viscousfluid and supported on a nonlinear elastic foundation.The proposed model is based on nonlocal Euler–Bernoulli beam theory.The Galerkin method and perturbation analysis are used to discretize the partial differential equation of motion and obtain the frequency-response equation,respectively.A detailed parametric study is reported into how the nonlocal parameter,foundation coefficients,fluid viscosity,and amplitude and frequency of the external force influence the nonlinear dynamics of the system.Subharmonic,quasi-periodic,and chaotic behaviors and hardening nonlinearity are revealed by means of the vibration time histories,frequency-response curves,bifurcation diagrams,phase portraits,power spectra,and Poincarémaps.Also,the results show that it is possible to eliminate irregular motion in the whole range of external force amplitude by selecting appropriate parameters.

Output Feedback Stabilization of High-Order Nonlinear Time-Delay Systems With Low-Order and High-Order Nonlinearities

Dear Editor,to This letter deals with the output feedback stabilization of a class of high-order nonlinear time-delay systems with more general low-order and high-order nonlinearities.By constructing reduced-order observer,based on homogeneous domination theory together with the adding a power integrator method,an output feedback controller is developed guarantee the equilibrium of the closed system globally uniformly asymptotically stable.

Nonlinear-Optical Analogies in Nuclear-Like Soliton Reactions:Selection Rules,Nonlinear Tunneling and Sub-Barrier Fusion–Fission

The main goal of our study is to reveal unexpected but intriguing analogies arising between optical solitons and nuclear physics,which still remain hidden from us.We consider the main cornerstones of the concept of nonlinear optics of nuclear reactions and the well-dressed repulsive-core solitons.On the base of this model,we reveal the most intriguing properties of the nonlinear tunneling of nucleus-like solitons and the soliton selfinduced sub-barrier transparency effect.We describe novel interesting and stimulating analogies between the interaction of nucleus-like solitons on the repulsive barrier and nuclear sub-barrier reactions.The main finding of this study concerns the conservation of total number of nucleons(or the baryon number)in nuclear-like soliton reactions.We show that inelastic interactions among well-dressed repulsive-core solitons arise only when a“cloud”of“dressing”spectral side-bands appears in the frequency spectra of the solitons.This property of nucleus-like solitons is directly related to the nuclear density distribution described by the dimensionless small shape-squareness parameter.Thus the Fourier spectra of nucleus-like solitons are similar to the nuclear form factors.We show that the nuclear-like reactions between well-dressed solitons are realized by“exchange”between“particle-like”side bands in their spectra.

Nonlinear Robust Stabilization of Ship Roll by Convex Optimization

Dear Editor,This letter presents a nonlinear robust controller design method for ship roll stabilization by combining the dual of Lyapunov's stability theorem with the sum of squares(SOS) technique. Varying initial metacentric height and ship speed are regarded as uncertainties, sea waves are considered as external disturbances, and then the robust nonlinear controller is designed. Taking a container ship as an example, simulations are performed to verify the effectiveness of the proposed design scheme.

Stability Analysis of a Class of Nonlinear Fractional Differential Systems With Riemann-Liouville Derivative

Dear Editor,This letter investigates the stability of n-dimensional nonlinear fractional differential systems with Riemann-Liouville derivative.By using the Mittag-Leffler function,Laplace transform and the Gronwall-Bellman lemma,one sufficient condition is attained for the asymptotical stability of a class of nonlinear fractional differential systems whose order lies in(0,2).According to this theory,if the nonlinear term satisfies some conditions,then the stability condition for nonlinear fractional differential systems is the same as the ones for corresponding linear systems.Two examples are provided to illustrate the applications of our result.

Higher-Order Nonlinear Effects on Optical Soliton Propagation and Their Interactions

When pursuing femtosecond-scale ultrashort pulse optical communication, one cannot overlook higher-order nonlinear effects. Based on the fundamental theoretical model of the variable coefficient coupled high-order nonlinear Schr¨odinger equation, we analytically explore the evolution of optical solitons in the presence of highorder nonlinear effects. Moreover, the interactions between two nearby optical solitons and their transmission in a nonuniform fiber are investigated. The stability of optical soliton transmission and interactions are found to be destroyed to varying degrees due to higher-order nonlinear effects. The outcomes may offer some theoretical references for achieving ultra-high energy optical solitons in the future.

Multiple mixed state variable incremental integration for reconstructing extreme multistability in a novel memristive hyperchaotic jerk system with multiple cubic nonlinearity

Memristor-based chaotic systems with infinite equilibria are interesting because they generate extreme multistability.Their initial state-dependent dynamics can be explained in a reduced-dimension model by converting the incremental integration of the state variables into system parameters.However,this approach cannot solve memristive systems in the presence of nonlinear terms other than the memristor term.In addition,the converted state variables may suffer from a degree of divergence.To allow simpler mechanistic analysis and physical implementation of extreme multistability phenomena,this paper uses a multiple mixed state variable incremental integration(MMSVII)method,which successfully reconstructs a four-dimensional hyperchaotic jerk system with multiple cubic nonlinearities except for the memristor term in a three-dimensional model using a clever linear state variable mapping that eliminates the divergence of the state variables.Finally,the simulation circuit of the reduced-dimension system is constructed using Multisim simulation software and the simulation results are consistent with the MATLAB numerical simulation results.The results show that the method of MMSVII proposed in this paper is useful for analyzing extreme multistable systems with multiple higher-order nonlinear terms.