Echo State Network With Probabilistic Regularization for Time Series Prediction

Recent decades have witnessed a trend that the echo state network(ESN)is widely utilized in field of time series prediction due to its powerful computational abilities.However,most of the existing research on ESN is conducted under the assumption that data is free of noise or polluted by the Gaussian noise,which lacks robustness or even fails to solve real-world tasks.This work handles this issue by proposing a probabilistic regularized ESN(PRESN)with robustness guaranteed.Specifically,we design a novel objective function for minimizing both the mean and variance of modeling error,and then a scheme is derived for getting output weights of the PRESN.Furthermore,generalization performance,robustness,and unbiased estimation abilities of the PRESN are revealed by theoretical analyses.Finally,experiments on a benchmark dataset and two real-world datasets are conducted to verify the performance of the proposed PRESN.The source code is publicly available at https://github.com/LongJinlab/probabilistic-regularized-echo-state-network.

Echo state network based symbol detection in chaotic baseband wireless communication

The Chaotic Baseband Wireless Communication System(CBWCS)is expected to eliminate the Inter-Symbol Interference(ISI)caused by multipath propagation by using the optimal decoding threshold that is the sum of the ISI caused by past decoded bits and the ISI caused by future transmitting bits.However,the current technique is only capable of removing partial effects of the ISI,because only past decoded bits are available for the suboptimal decoding threshold calculation.The unavailability of the future information needed for the optimal decoding threshold is an obstacle to further improve the Bit Error Rate(BER)performance.In contrast to the previous method using Echo State Network(ESN)to predict one future bit,the proposed method in this paper predicts the optimal decoding threshold directly using ESN.The proposed ESN-based threshold prediction method simplifies the symbol decoding operation by avoiding the iterative prediction of the output waveform points using ESN and accumulated error caused by the iterative operation.With this approach,the calculation complexity is reduced compared to the previous ESN-based approach.The proposed method achieves better BER performance compared to the previous method.The reason for this superior result is twofold.First,the proposed ESN is capable of using more future symbols information conveyed by the ESN input to obtain more accurate threshold rather than the previous method in which only one future symbol was available.Second,the proposed method here does not need to estimate the channel information using Least Squared(LS)method,which avoids the extra error caused by inaccurate channel information estimation.Simulation results and experiment based on a wireless open-access research platform under a practical wireless channel show the effectiveness and superiority of the proposed method.

Stock Price Forecasting: An Echo State Network Approach

Forecasting stock prices using deep learning models suffers from pro-blems such as low accuracy,slow convergence,and complex network structures.This study developed an echo state network(ESN)model to mitigate such pro-blems.We compared our ESN with a long short-term memory(LSTM)network by forecasting the stock data of Kweichow Moutai,a leading enterprise in China’s liquor industry.By analyzing data for 120,240,and 300 days,we generated fore-cast data for the next 40,80,and 100 days,respectively,using both ESN and LSTM.In terms of accuracy,ESN had the unique advantage of capturing non-linear data.Mean absolute error(MAE)was used to present the accuracy results.The MAEs of the data forecast by ESN were 0.024,0.024,and 0.025,which were,respectively,0.065,0.007,and 0.009 less than those of LSTM.In terms of con-vergence,ESN has a reservoir state-space structure,which makes it perform faster than other models.Root-mean-square error(RMSE)was used to present the con-vergence time.In our experiment,the RMSEs of ESN were 0.22,0.27,and 0.26,which were,respectively,0.08,0.01,and 0.12 less than those of LSTM.In terms of network structure,ESN consists only of input,reservoir,and output spaces,making it a much simpler model than the others.The proposed ESN was found to be an effective model that,compared to others,converges faster,forecasts more accurately,and builds time-series analyses more easily.

A Prediction Method Based on Improved Echo State Network for COVID-19 Nonlinear Time Series

This paper proposes a prediction method based on improved Echo State Network for COVID-19 nonlinear time series, which improves the Echo State Network from the reservoir topology and the output weight matrix, and adopt the ABC (Artificial Bee Colony) algorithm based on crossover and crowding strategy to optimize the parameters. Finally, the proposed method is simulated and the results show that it has stronger prediction ability for COVID-19 nonlinear time series.

Analysis of prediction performance in wavelet minimum complexity echo state network

Echo state network （ESN） has become one of the most popular recurrent neural networks （RNN） for its good prediction performance of non-linear time series and simple training process. But several problems still prevent ESN from becoming a widely used tool. The most prominent problem is its high complexity with lots of random parameters. Aiming at this problem, a minimum complexity ESN model （MCESN） was proposed. In this paper, we proposed a new wavelet minimum complexity ESN model （WMCESN） to improve the prediction accuracy and increase the practical applicability. Our new model inherits the characters of minimum complexity ESN model using the fixed parameters and simple circle topology. We injected wavelet neurons to replace the original neurons in internal reservoir and designed a wavelet parameter matrix to reduce the computing time. By using different datasets, our new model performed better than the minimum complexity ESN model with normal neurons, but only utilized tiny time cost. We also used our own packets of transmission control protocol （TCP） and user datagram protocol （UDP） dataset to prove that our model can deal with the data packet bit prediction problem well.

Hierarchy echo state network based service-awareness in 10G-EPON

Aimed to enhance the supporting ability for diversified services, this paper proposes a hierarchy echo state network（HESE） based service-awareness（SA）（HESN-SA） mechanism in 10 Gbit/s Ethernet passive optical network（10G-EPON）. In this HESN-SA, hierarchy architecture is adopted to realize echo state network（ESN） classification based SA. According to the network architecture of 10G-EPON, the parent-ESN（p-ESN） module works in the optical line terminal（OLT）, while the sub-ESN（s-ESN） module is embedded in optical network units（ONUs）. Thus, the p-ESN plays the main function of SA with a total view of this system, and s-ESN in each ONU conducts the SA function under the control of p-ESN. Thus, resources allocation and transport policy are both determined by the proposed mechanism through cooperation between OLT and ONUs. Simulation results show that the HESN-SA can improve the supporting ability for multiple services.

基于Echo State Neural Networks的短期交通流预测算法

在城市交通环境,交通流的正确预测是比较困难,因为多个十字路口,这使得预置的交通控制模型之间的相互作用和intertwinement不能保持始终高性能在所有的交通情况。

Simplified Echo-State-Network Based Services Awareness for High-Speed Passive Optical Network

With the challenge from services diversity grows greatly,the service-oriented supporting ability is required to current high-speed passive optical network(PON) .Aimed to enhance the quality of service(Qo S) brought by diversified-services,this paper proposes an Simplified Echo State Network(SESN) Based Services Awareness scheme in High-Speed PON(Passive Optical Network) .In this proposed scheme,the ring topology is adopted in the reservoir of SESN to reduce the complexity of original Echo State Network,and system dynamics equation is introduced to keep the accuracy of SESN.According to the network architecture of 10G-EPON,a SESN Master is running in the OLT and a number of SESN Agents work in ONUs.The SESN Master plays the main function of service-awareness from the total view of various kinds services in 10G-EPON system,by fully SESN training.Then,the reservoir information of well-trained SESN in OLT will be broadcasted to all ONUs and those SESN Agents working in ONUs are allowed to conducts independent service-awareness function.Thus,resources allocation and transport policy are both determined just only in ONUs.Simulation results show that the proposed mechanism is able to better supporting ability for multiple services.

Modeling deterministic echo state network with loop reservoir

Echo state network (ESN), which efficiently models nonlinear dynamic systems, has been proposed as a special form of recurrent neural network. However, most of the proposed ESNs consist of complex reservoir structures, leading to excessive computational cost. Recently, minimum complexity ESNs were proposed and proved to exhibit high performance and low computational cost. In this paper, we propose a simple deterministic ESN with a loop reservoir, i.e., an ESN with an adjacent-feedback loop reservoir. The novel reservoir is constructed by introducing regular adjacent feedback based on the simplest loop reservoir. Only a single free parameter is tuned, which considerably simplifies the ESN construction. The combination of a simplified reservoir and fewer free parameters provides superior prediction performance. In the benchmark datasets and real-world tasks, our scheme obtains higher prediction accuracy with relatively low complexity, compared to the classic ESN and the minimum complexity ESN. Furthermore, we prove that all the linear ESNs with the simplest loop reservoir possess the same memory capacity, arbitrarily converging to the optimal value.

Improved vocal effort modeling by exploiting echo state network and radial basis function network

The independent hypothesis between frames in vocal effect(VE) recognition makes it difficult for frame based spectral features to describe the intrinsic temporal correlation and dynamic change information in speech phenomena. A novel VE detection method based on echo state network(ESN) is proposed. The input sequences are mapped into a fixed-dimensionality vector in high dimensional coding space by reservoir of the ESN. Then, radial basis function(RBF) networks are employed to fit the probability density function(pdf) of each VE mode by using the vectors in the high dimensional coding space. Finally, the minimum error rate Bayesian decision is employed to judge the VE mode. The experiments which are conducted on isolated words test set achieve 79.5% average recognition accuracy, and the results show that the proposed method can overcome the defect of the independent hypothesis between frames effectively.

Echo-state-network classification based multi-services awareness in high-speed optical passive networks

With the challenge of great growing of services diversity,service-oriented supporting ability is required by current high-speed passive optical network( PON). Aimed at enhancing the quality of service( Qo S) brought by diversified-services,this study proposes an echo state network( ESN)based multi-service awareness mechanism in 10-Gigabite ethernet passive optical network( 10GEPON). In the proposed approach,distributed architecture is adopted to realize this ESN based multi-service awareness. According to the network architecture of 10G-EPON,where a main ESN is running in OLT and a number of ESN agents works in ONUs. The main-ESN plays the main function of service-awareness from the total view of various kinds of services in 10G-EPON system,by full ESN training. Then,the reservoir information of well-trained ESN in OLT will be broadcasted to all ONUs and those ESN agents working in ONUs are allowed to conduct independent service-awareness function. Thus,resources allocation and transport policy are both determined only in ONUs. Simulation results show that the proposed mechanism is able to better support the ability of multiple services.

Lightweight Intrusion Detection Using Reservoir Computing

The blockchain-empowered Internet of Vehicles(IoV)enables various services and achieves data security and privacy,significantly advancing modern vehicle systems.However,the increased frequency of data transmission and complex network connections among nodes also make them more susceptible to adversarial attacks.As a result,an efficient intrusion detection system(IDS)becomes crucial for securing the IoV environment.Existing IDSs based on convolutional neural networks(CNN)often suffer from high training time and storage requirements.In this paper,we propose a lightweight IDS solution to protect IoV against both intra-vehicle and external threats.Our approach achieves superior performance,as demonstrated by key metrics such as accuracy and precision.Specifically,our method achieves accuracy rates ranging from 99.08% to 100% on the Car-Hacking dataset,with a remarkably short training time.

偏置剪枝叠式自编码回声状态网络的时序预测

针对大多数模型对时间序列预测数据的预测准确率较低,为提升时间序列的预测精度,提出一种基于Biased Drop-weight的偏置剪枝叠式自编码回声状态网络(BD-AE-SGESN)的深度模型。以叠式ESN为多层深度网络框架,提出一种生成式AE算法生成每一层的输入权值,利用BD算法根据输入权重激活值进行剪枝。对比实验结果表明,该模型能够有效提升预测准确率,在3个不同的数据上,相比其它模型有着较小的预测误差和较高的稳定度。

基于知识与AW-ESN融合的烧结过程FeO含量预测

氧化亚铁(FeO)含量是衡量烧结矿强度和还原性的重要指标,烧结过程FeO含量的实时准确预测对于提升烧结质量、优化烧结工艺具有重要意义.然而烧结过程热状态参数缺失、过程参数波动频繁给FeO含量的高精度预测带来巨大的挑战,为此,提出一种基于知识与变权重回声状态网络融合(Fusion of data-knowledge and adaptive weight echo state network, DK-AWESN)的烧结过程FeO含量预测方法.首先,针对烧结过程热状态参数缺失的问题,建立烧结料层最高温度分布模型,实现基于料层温度分布特征的FeO含量等级划分;其次,针对烧结过程参数波动频繁的问题,提出基于核函数高维映射的多尺度数据配准方法,有效抑制离群点的影响,提升建模数据的质量;最后,针对烧结过程数据驱动模型缺乏机理认知致使模型预测精度不高的问题,将过程数据中提取得到的FeO含量等级知识与AW-ESN (Adaptive weight echo state network)结合,建立DK-AWESN模型,有效提升复杂工况下FeO含量的预测精度.现场工业数据试验表明,所提方法能实时准确地预测烧结过程FeO含量,为烧结过程的智能化调控提供实时有效的FeO含量反馈信息.

基于EMD-DESN的无人机集群航迹目的地预测

无人机(unmanned aerial vehicle,UAV)集群作战样式多样、运动模式复杂,导致集群航迹目的地难以预测。为解决上述问题,本文提出了一种基于经验模态分解(empirical mode decomposition,EMD)和深度回声状态网络(deep echo state network,DESN)的UAV集群航迹目的地预测算法。为使集群运动模型更真实地模拟UAV集群作战过程,本文引入航向误差时变方差,改进了Olfati-Saber集群运动模型的虚拟领导项。为处理因群内的协同作用和集群航向误差导致的运动非平稳性,引入了EMD,对UAV航迹序列进行重构。考虑到获知航迹的时序性,设计了滑窗结构,采用DESN对重构航迹的不同时段进行目的地预测。仿真实验结果表明,本文提出的EMD-DESN算法较基本DESN算法能以更高的准确度预测UAV集群航迹目的地,并能更早地实现稳定的正确预测。

基于双通道回声状态网络的时间序列补全及单步预测

随着物联网的发展,众多传感器采集到大量具有丰富数据相关性的时间序列,为各种数据挖掘应用提供强大的数据支持。然而,一些客观或主观原因(如设备故障、稀疏感知等)往往会造成采集到的数据出现不同程度的缺失。虽然已有很多方法被提出用于解决这一问题,但这些方法在数据相关性方面或考虑不够全面,或计算成本过高。而且,现有方法仅关注对缺失值的补全,未能兼顾下游应用。针对上述不足,设计了一种兼顾补全与预测任务的双通道回声状态网络。两个通道的网络虽共用输入层,但具有各自的储备池和输出层。两者最大的区别是左/右通道的输出层分别表示输入层前/后一个时刻对应的目标值或预补值。最后将两个通道的估计值进行融合,充分利用来自缺失时刻之前和之后的数据相关性以进一步提升性能。两种缺失现象下(随机缺失和分段缺失)不同缺失率的实验结果表明,所提模型无论是在补全精度还是预测精度上都优于目前流行的各类方法。

储能用质子交换膜燃料电池长期老化预测

质子交换膜燃料电池(PEMFC)的长期老化预测有助于缩短耐久性测试时间,降低成本,为维护策略提供依据。针对超参数问题,提出一种将优化算法和储备池计算相结合的数据驱动预测方法。基于耐久性测试数据集,以电堆输出电压为老化指标,利用麻雀搜索算法(SSA)优化回声状态网络(ESN)的储备池尺寸、泄漏率和正则化系数,以构建预测模型。分别利用原始数据的前30%、40%、50%和60%作为训练集训练模型,验证模型在各训练集比例下的长期老化预测性能。训练集比例为30%时,所提方法在静态工况下的长期预测均方根误差(RMSE)达到0.0083,准动态工况下可达到0.0359。

基于PLESN和LESQRN概率预测模型的短期电力负荷预测

针对现有电力负荷预测不能很好反映负荷数据的周期性和趋势性以及残差的波动性特征提出一种考虑周期性建模的泄露积分型回声状态网络点预测模型和泄露积分型回声状态分位数回归网络概率预测模型组合的短期电力负荷预测方法.首先为了捕捉负荷的多重特征定义了周期性和趋势性损失函数辅助优化点预测模型然后为克服残差的波动问题利用概率预测模型对点预测值与真实值的残差进行建模预测最后整合同时刻的点预测值与残差预测区间得到概率预测模型结果.实际算例结果表明与其他模型相比所提模型不仅有效抑制尖端振荡现象而且能够生成可靠的概率密度分布.

Intelligent Passive Detection of Aerial Target in Space-Air-Ground Integrated Networks

Passive detection of moving target is an important part of intelligent surveillance. Satellite has the potential to play a key role in many applications of space-air-ground integrated networks(SAGIN). In this paper, we propose a novel intelligent passive detection method for aerial target based on reservoir computing networks. Specifically, delayed feedback networks are utilized to refine the direct signals from the satellite in the reference channels. In addition, the satellite direct wave interference in the monitoring channels adopts adaptive interference suppression using the minimum mean square error filter. Furthermore, we employ decoupling echo state networks to predict the clutter interference in the monitoring channels and construct the detection statistics accordingly. Finally, a multilayer perceptron is adopted to detect the echo signal after interference suppression. Extensive simulations is conducted to evaluate the performance of our proposed method. Results show that the detection probability is almost 100% when the signal-to-interference ratio of echo signal is-36 dB, which demonstrates that our proposed method achieves efficient passive detection for aerial targets in typical SAGIN scenarios.

具有双储层结构的动态误差补偿回声状态网络

针对传统回声状态网络难以有效应对高阶非线性复杂模型问题,本文在理论分析的基础上提出了一种双储层结构的误差补偿回声状态网络,并设计了该网络的学习算法.该网络由计算层和补偿层构成,计算层主要承担拟合任务,补偿层则作为状态跟随器,实时补偿由于计算层对期望方差估计不足而导致的幅值偏差.对多阶振荡器和真实高阶非线性数据集的实验结果表明,本文所提网络结构较常规网络具有更高的稳定性和泛化性能,尤其对高阶非线性复杂模型的预测精度大幅度提升.