Echo State Network With Probabilistic Regularization for Time Series Prediction

Recent decades have witnessed a trend that the echo state network(ESN)is widely utilized in field of time series prediction due to its powerful computational abilities.However,most of the existing research on ESN is conducted under the assumption that data is free of noise or polluted by the Gaussian noise,which lacks robustness or even fails to solve real-world tasks.This work handles this issue by proposing a probabilistic regularized ESN(PRESN)with robustness guaranteed.Specifically,we design a novel objective function for minimizing both the mean and variance of modeling error,and then a scheme is derived for getting output weights of the PRESN.Furthermore,generalization performance,robustness,and unbiased estimation abilities of the PRESN are revealed by theoretical analyses.Finally,experiments on a benchmark dataset and two real-world datasets are conducted to verify the performance of the proposed PRESN.The source code is publicly available at https://github.com/LongJinlab/probabilistic-regularized-echo-state-network.

Echo state network based symbol detection in chaotic baseband wireless communication

The Chaotic Baseband Wireless Communication System(CBWCS)is expected to eliminate the Inter-Symbol Interference(ISI)caused by multipath propagation by using the optimal decoding threshold that is the sum of the ISI caused by past decoded bits and the ISI caused by future transmitting bits.However,the current technique is only capable of removing partial effects of the ISI,because only past decoded bits are available for the suboptimal decoding threshold calculation.The unavailability of the future information needed for the optimal decoding threshold is an obstacle to further improve the Bit Error Rate(BER)performance.In contrast to the previous method using Echo State Network(ESN)to predict one future bit,the proposed method in this paper predicts the optimal decoding threshold directly using ESN.The proposed ESN-based threshold prediction method simplifies the symbol decoding operation by avoiding the iterative prediction of the output waveform points using ESN and accumulated error caused by the iterative operation.With this approach,the calculation complexity is reduced compared to the previous ESN-based approach.The proposed method achieves better BER performance compared to the previous method.The reason for this superior result is twofold.First,the proposed ESN is capable of using more future symbols information conveyed by the ESN input to obtain more accurate threshold rather than the previous method in which only one future symbol was available.Second,the proposed method here does not need to estimate the channel information using Least Squared(LS)method,which avoids the extra error caused by inaccurate channel information estimation.Simulation results and experiment based on a wireless open-access research platform under a practical wireless channel show the effectiveness and superiority of the proposed method.

Echo-state-network classification based multi-services awareness in high-speed optical passive networks

With the challenge of great growing of services diversity,service-oriented supporting ability is required by current high-speed passive optical network( PON). Aimed at enhancing the quality of service( Qo S) brought by diversified-services,this study proposes an echo state network( ESN)based multi-service awareness mechanism in 10-Gigabite ethernet passive optical network( 10GEPON). In the proposed approach,distributed architecture is adopted to realize this ESN based multi-service awareness. According to the network architecture of 10G-EPON,where a main ESN is running in OLT and a number of ESN agents works in ONUs. The main-ESN plays the main function of service-awareness from the total view of various kinds of services in 10G-EPON system,by full ESN training. Then,the reservoir information of well-trained ESN in OLT will be broadcasted to all ONUs and those ESN agents working in ONUs are allowed to conduct independent service-awareness function. Thus,resources allocation and transport policy are both determined only in ONUs. Simulation results show that the proposed mechanism is able to better support the ability of multiple services.

Simplified Echo-State-Network Based Services Awareness for High-Speed Passive Optical Network

With the challenge from services diversity grows greatly,the service-oriented supporting ability is required to current high-speed passive optical network(PON) .Aimed to enhance the quality of service(Qo S) brought by diversified-services,this paper proposes an Simplified Echo State Network(SESN) Based Services Awareness scheme in High-Speed PON(Passive Optical Network) .In this proposed scheme,the ring topology is adopted in the reservoir of SESN to reduce the complexity of original Echo State Network,and system dynamics equation is introduced to keep the accuracy of SESN.According to the network architecture of 10G-EPON,a SESN Master is running in the OLT and a number of SESN Agents work in ONUs.The SESN Master plays the main function of service-awareness from the total view of various kinds services in 10G-EPON system,by fully SESN training.Then,the reservoir information of well-trained SESN in OLT will be broadcasted to all ONUs and those SESN Agents working in ONUs are allowed to conducts independent service-awareness function.Thus,resources allocation and transport policy are both determined just only in ONUs.Simulation results show that the proposed mechanism is able to better supporting ability for multiple services.

Stock Price Forecasting: An Echo State Network Approach

Forecasting stock prices using deep learning models suffers from pro-blems such as low accuracy,slow convergence,and complex network structures.This study developed an echo state network(ESN)model to mitigate such pro-blems.We compared our ESN with a long short-term memory(LSTM)network by forecasting the stock data of Kweichow Moutai,a leading enterprise in China’s liquor industry.By analyzing data for 120,240,and 300 days,we generated fore-cast data for the next 40,80,and 100 days,respectively,using both ESN and LSTM.In terms of accuracy,ESN had the unique advantage of capturing non-linear data.Mean absolute error(MAE)was used to present the accuracy results.The MAEs of the data forecast by ESN were 0.024,0.024,and 0.025,which were,respectively,0.065,0.007,and 0.009 less than those of LSTM.In terms of con-vergence,ESN has a reservoir state-space structure,which makes it perform faster than other models.Root-mean-square error(RMSE)was used to present the con-vergence time.In our experiment,the RMSEs of ESN were 0.22,0.27,and 0.26,which were,respectively,0.08,0.01,and 0.12 less than those of LSTM.In terms of network structure,ESN consists only of input,reservoir,and output spaces,making it a much simpler model than the others.The proposed ESN was found to be an effective model that,compared to others,converges faster,forecasts more accurately,and builds time-series analyses more easily.

A Prediction Method Based on Improved Echo State Network for COVID-19 Nonlinear Time Series

This paper proposes a prediction method based on improved Echo State Network for COVID-19 nonlinear time series, which improves the Echo State Network from the reservoir topology and the output weight matrix, and adopt the ABC (Artificial Bee Colony) algorithm based on crossover and crowding strategy to optimize the parameters. Finally, the proposed method is simulated and the results show that it has stronger prediction ability for COVID-19 nonlinear time series.

Intelligent Passive Detection of Aerial Target in Space-Air-Ground Integrated Networks

Passive detection of moving target is an important part of intelligent surveillance. Satellite has the potential to play a key role in many applications of space-air-ground integrated networks(SAGIN). In this paper, we propose a novel intelligent passive detection method for aerial target based on reservoir computing networks. Specifically, delayed feedback networks are utilized to refine the direct signals from the satellite in the reference channels. In addition, the satellite direct wave interference in the monitoring channels adopts adaptive interference suppression using the minimum mean square error filter. Furthermore, we employ decoupling echo state networks to predict the clutter interference in the monitoring channels and construct the detection statistics accordingly. Finally, a multilayer perceptron is adopted to detect the echo signal after interference suppression. Extensive simulations is conducted to evaluate the performance of our proposed method. Results show that the detection probability is almost 100% when the signal-to-interference ratio of echo signal is-36 dB, which demonstrates that our proposed method achieves efficient passive detection for aerial targets in typical SAGIN scenarios.

Lightweight Intrusion Detection Using Reservoir Computing

The blockchain-empowered Internet of Vehicles(IoV)enables various services and achieves data security and privacy,significantly advancing modern vehicle systems.However,the increased frequency of data transmission and complex network connections among nodes also make them more susceptible to adversarial attacks.As a result,an efficient intrusion detection system(IDS)becomes crucial for securing the IoV environment.Existing IDSs based on convolutional neural networks(CNN)often suffer from high training time and storage requirements.In this paper,we propose a lightweight IDS solution to protect IoV against both intra-vehicle and external threats.Our approach achieves superior performance,as demonstrated by key metrics such as accuracy and precision.Specifically,our method achieves accuracy rates ranging from 99.08% to 100% on the Car-Hacking dataset,with a remarkably short training time.

Short-term prediction of photovoltaic power generation based on LMD-EE-ESN with error correction

Considering the instability of the output power of photovoltaic(PV)generation system,to improve the power regulation ability of PV power during grid-connected operation,based on the quantitative analysis of meteorological conditions,a short-term prediction method of PV power based on LMD-EE-ESN with iterative error correction was proposed.Firstly,through the fuzzy clustering processing of meteorological conditions,taking the power curves of PV power generation in sunny,rainy or snowy,cloudy,and changeable weather as the reference,the local mean decomposition(LMD)was carried out respectively,and their energy entropy(EE)was taken as the meteorological characteristics.Then,the historical generation power series was decomposed by LMD algorithm,and the hierarchical prediction of the power curve was realized by echo state network(ESN)prediction algorithm combined with meteorological characteristics.Finally,the iterative error theory was applied to the correction of power prediction results.The analysis of the historical data in the PV power generation system shows that this method avoids the influence of meteorological conditions in the short-term prediction of PV output power,and improves the accuracy of power prediction on the condition of hierarchical prediction and iterative error correction.

Improvement of Shape Recognition Performance of Sendzimir Mill Control Systems Using Echo State Neural Networks

High rigidity twenty-high Sendzimir mills （ZRMs） are widely used for rolling stainless steels, silicon sheets, etc. A ZRM uses a small diameter work roll to produce massive rolling forces. Since a work roll with a small diameter can be bent easily, strips often have complex shapes with mixed quarter and deep edge waves in the shape of plates. In order to solve this problem, fuzzy neural network controls are generally used for shape： recognition in ZRM control systems. Among various neural network types, the multi-layer perceptron （MLP） is typically used in current ZRMs. However, an MLP causes the loss of a large amount of shape recognition data. To improve the shape recognition per- formance of ZRM control systems, echo state networks （ESNs） are proposed to be used. Through simulation re- sults, it is found that shape recognition performance could be improved using the proposed ESN method.

Modeling deterministic echo state network with loop reservoir

Echo state network (ESN), which efficiently models nonlinear dynamic systems, has been proposed as a special form of recurrent neural network. However, most of the proposed ESNs consist of complex reservoir structures, leading to excessive computational cost. Recently, minimum complexity ESNs were proposed and proved to exhibit high performance and low computational cost. In this paper, we propose a simple deterministic ESN with a loop reservoir, i.e., an ESN with an adjacent-feedback loop reservoir. The novel reservoir is constructed by introducing regular adjacent feedback based on the simplest loop reservoir. Only a single free parameter is tuned, which considerably simplifies the ESN construction. The combination of a simplified reservoir and fewer free parameters provides superior prediction performance. In the benchmark datasets and real-world tasks, our scheme obtains higher prediction accuracy with relatively low complexity, compared to the classic ESN and the minimum complexity ESN. Furthermore, we prove that all the linear ESNs with the simplest loop reservoir possess the same memory capacity, arbitrarily converging to the optimal value.

Wheel-Individual Estimation of the Friction Potential for Split Friction and Changing Friction Conditions for the Application in an Automated Emergency Braking System

Including information of the current road surface conditions can significantly improve the effectiveness of an AEB （automated emergency braking） system to avoid accidents or reduce the injury severity in rear-end crashes. A method to estimate the friction potential based on on-board sensor information is shown in this work. This work expands the scope of existing investigations on whether the accuracy needed for the warning and intervention strategies of AEB can be reached with the proposed method. First, the bandwidth of surface conditions investigated is extended by including low friction surfaces comparable to ice. Second, situations of changing surface conditions and wheel-individual surface conditions were evaluated. Finally, estimation based on different sensor sets was conducted with regard to series application. The investigations are based on measurements performed on a proving ground. The main emphasis was placed on estimation during longitudinal driving conditions. The used sensors include advanced vehicle dynamics measurement equipment as well as standard on-board sensors of the vehicle.

Analysis of prediction performance in wavelet minimum complexity echo state network

Echo state network （ESN） has become one of the most popular recurrent neural networks （RNN） for its good prediction performance of non-linear time series and simple training process. But several problems still prevent ESN from becoming a widely used tool. The most prominent problem is its high complexity with lots of random parameters. Aiming at this problem, a minimum complexity ESN model （MCESN） was proposed. In this paper, we proposed a new wavelet minimum complexity ESN model （WMCESN） to improve the prediction accuracy and increase the practical applicability. Our new model inherits the characters of minimum complexity ESN model using the fixed parameters and simple circle topology. We injected wavelet neurons to replace the original neurons in internal reservoir and designed a wavelet parameter matrix to reduce the computing time. By using different datasets, our new model performed better than the minimum complexity ESN model with normal neurons, but only utilized tiny time cost. We also used our own packets of transmission control protocol （TCP） and user datagram protocol （UDP） dataset to prove that our model can deal with the data packet bit prediction problem well.

Hierarchy echo state network based service-awareness in 10G-EPON

Aimed to enhance the supporting ability for diversified services, this paper proposes a hierarchy echo state network（HESE） based service-awareness（SA）（HESN-SA） mechanism in 10 Gbit/s Ethernet passive optical network（10G-EPON）. In this HESN-SA, hierarchy architecture is adopted to realize echo state network（ESN） classification based SA. According to the network architecture of 10G-EPON, the parent-ESN（p-ESN） module works in the optical line terminal（OLT）, while the sub-ESN（s-ESN） module is embedded in optical network units（ONUs）. Thus, the p-ESN plays the main function of SA with a total view of this system, and s-ESN in each ONU conducts the SA function under the control of p-ESN. Thus, resources allocation and transport policy are both determined by the proposed mechanism through cooperation between OLT and ONUs. Simulation results show that the HESN-SA can improve the supporting ability for multiple services.

Constrained docking control for shipborne UAV SideArm recovery

In this work,we sought to investigate constrained docking control during shipborne SideArm recovery of an Unmanned Aerial Vehicle(UAV)under preassigned safe docking constraints,rough ocean environments,and different initial positions.The aim was to solve the UAV tracking-lag problem that manifests when attempting to dock with a rapidly moving SideArm and to improve the accuracy and rapidity of docking.First,together with the formulations of the shipborne SideArm system and environmental airflows,the affine nonlinear dynamics of the hook was established to reduce tracking lag.Then,echo state network approximators with good approximation capacity and low computational consumption were designed to accurately approximate the UAV’s unknown nonlinear dynamics.With feedforward compensation provided by these approximators,a nonlinear-mapping-based constrained docking control law was developed for shipborne SideArm recovery of UAVs.This approach to controlling the docking trajectory and the forward docking speed of the UAV can achieve rapid and exact docking with a moving SideArm,without violating the preassigned safe docking-constraint envelopes.Simulations under different docking scenarios were used to validate the effectiveness and advantages of the proposed docking-control algorithm.

A Hybrid Time-delay Prediction Method for Networked Control System

This paper presents an Ethernet based hybrid method for predicting random time-delay in the networked control system.First,db3 wavelet is used to decompose and reconstruct time-delay sequence,and the approximation component and detail components of time-delay sequences are fgured out.Next,one step prediction of time-delay is obtained through echo state network(ESN)model and auto-regressive integrated moving average model(ARIMA)according to the diferent characteristics of approximate component and detail components.Then,the fnal predictive value of time-delay is obtained by summation.Meanwhile,the parameters of echo state network is optimized by genetic algorithm.The simulation results indicate that higher accuracy can be achieved through this prediction method.

An extended SHESN with leaky integrator neuron and inhibitory connection for Mackey-Glass prediction

Echo state network （ESN） proposed by Jaeger in 2001 has remarkable capabilities of approximating dynamics for complex systems, such as Mackey-Glass problem. Compared to that of ESN, the scale-free highlyclustered ESN, i.e., SHESN, which state reservoir has both small-world phenomenon and scale-free feature, exhibits even stronger approximation capabilities of dynamics and better echo state property. In this paper, we extend the state reservoir of SHESN using leaky integrator neurons and inhibitory connections, inspired from the advances in neurophysiology. We apply the extended SHESN, called eSHESN, to the Mackey-Glass prediction problem. The experimental results show that the e-SHESN considerably outperforms the SHESN in prediction capabilities of the Mackey-Glass chaotic time-series. Meanwhile, the interesting complex network characteristic in the state reservoir, including the small-world property and the scale-free feature, remains unchanged. In addition, we unveil that the original SHESN may be unstable in some cases. However, the proposed e-SHESN model is shown to be able to address the flaw through the enhancement of the network stability. Specifically, by using the ridge regression instead of the linear regression, the stability of e-SHESN could be much more largely improved.

Deep learning-based symbol detection algorithm in IMDD-OOFDM system

In the current research on intensity-modulation and direct-detection optical orthogonal frequency division multiplexing(IMDD-OOFDM) system, effective channel compensation is a key factor to improve system performance. In order to improve the efficiency of channel compensation, a deep learning-based symbol detection algorithm is proposed in this paper for IMDD-OOFDM system. Firstly, a high-speed data streams symbol synchronization algorithm based on a training sequence is used to ensure accurate symbol synchronization. Then the traditional channel estimation and channel compensation are replaced by an echo state network(ESN) to restore the transmitted signal. Finally, we collect the data from the system experiment and calculate the signal-to-noise ratio(SNR). The analysis of the SNR optimized by the ESN proves that the ESN-based symbol detection algorithm is effective in compensating nonlinear distortion.

RLSESN-based PID adaptive control for a novel wearable rehabilitation robotic hand driven by PM-TS actuators

Purpose-The purpose of this paper is to develop a novel wearable rehabilitation robotic hand driven by Pneumatic Muscle-Torsion Spring(PM-TS)for finger therapy.PM has complex nonlinear dynamics,which makes PM modelling difficult.To realize high-accurate tracking for the robotic hand,an Echo State Network(ESN)-based PID adaptive controller is proposed,even though the plant model is unknown.Design/methodology/approach-To drive a single joint of rehabilitation robotic hand,the paper proposes a new PM-TS actuator comprising a Pneumatic Muscle(PM)and a Torsion Spring(TS).Based on the novel actuator,a wearable robotic hand is designed.By employing the model-free approximation capability of ESN,the RLSESN based PID adaptive controller is presented for improving the trajectory tracking performance of the rehabilitation robotic hand.An ESN together with Recursive Least Square(RLS)is called a RLSESN,where the ESN output weight matrix is updated by the online RLS learning algorithm.Findings–Practical experiments demonstrate the validity of the PM-TS actuator and indicate that the performance of the RLSESN based PID adaptive controller is better than that of the conventional PID controller.In addition,they also verify the effectiveness of the proposed rehabilitation robotic hand.Originality/value–A new PM-TS actuator configuration that uses a PM and a torsion spring for bi-directional movement of joint is presented.By utilizing the new PM-TS actuator,a novel wearable rehabilitation robotic hand for finger therapy is designed.Based on the unknown plant model,the RLSESN_PID controller is proposed to attain satisfactory performance.