Echo State Network With Probabilistic Regularization for Time Series Prediction

Recent decades have witnessed a trend that the echo state network(ESN)is widely utilized in field of time series prediction due to its powerful computational abilities.However,most of the existing research on ESN is conducted under the assumption that data is free of noise or polluted by the Gaussian noise,which lacks robustness or even fails to solve real-world tasks.This work handles this issue by proposing a probabilistic regularized ESN(PRESN)with robustness guaranteed.Specifically,we design a novel objective function for minimizing both the mean and variance of modeling error,and then a scheme is derived for getting output weights of the PRESN.Furthermore,generalization performance,robustness,and unbiased estimation abilities of the PRESN are revealed by theoretical analyses.Finally,experiments on a benchmark dataset and two real-world datasets are conducted to verify the performance of the proposed PRESN.The source code is publicly available at https://github.com/LongJinlab/probabilistic-regularized-echo-state-network.

Echo state network based symbol detection in chaotic baseband wireless communication

The Chaotic Baseband Wireless Communication System(CBWCS)is expected to eliminate the Inter-Symbol Interference(ISI)caused by multipath propagation by using the optimal decoding threshold that is the sum of the ISI caused by past decoded bits and the ISI caused by future transmitting bits.However,the current technique is only capable of removing partial effects of the ISI,because only past decoded bits are available for the suboptimal decoding threshold calculation.The unavailability of the future information needed for the optimal decoding threshold is an obstacle to further improve the Bit Error Rate(BER)performance.In contrast to the previous method using Echo State Network(ESN)to predict one future bit,the proposed method in this paper predicts the optimal decoding threshold directly using ESN.The proposed ESN-based threshold prediction method simplifies the symbol decoding operation by avoiding the iterative prediction of the output waveform points using ESN and accumulated error caused by the iterative operation.With this approach,the calculation complexity is reduced compared to the previous ESN-based approach.The proposed method achieves better BER performance compared to the previous method.The reason for this superior result is twofold.First,the proposed ESN is capable of using more future symbols information conveyed by the ESN input to obtain more accurate threshold rather than the previous method in which only one future symbol was available.Second,the proposed method here does not need to estimate the channel information using Least Squared(LS)method,which avoids the extra error caused by inaccurate channel information estimation.Simulation results and experiment based on a wireless open-access research platform under a practical wireless channel show the effectiveness and superiority of the proposed method.

基于TCN-BiLSTM-Attention-ESN的光伏功率预测

针对光伏发电功率随机性强、难以准确预测的问题,提出一种基于时间卷积网络(TCN)、双向长短期记忆网络(BiLSTM)和回声状态网络(ESN)的组合预测方法。首先,使用自适应噪声完备集合经验模态分解(CEEMDAN)将功率数据分解为一系列相对平稳、不同波动模式的子功率序列;再将分解重构后的功率序列和其他特征序列输入到TCN-BiLSTM-Attention-ESN组合模型中,其中TCN-BiLSTM-Attention用于提取光伏序列波动特征并构建时空特征向量;最后,将所提取的时空特征向量输入ESN获得预测结果。采用新疆某光伏电站的光伏功率数据进行验证,结果表明与时下先进的预测方法相比,所提方法具有更高的预测精度,有助于提升光伏发电占比,保障电力系统平衡和运行安全。

基于知识与AW-ESN融合的烧结过程FeO含量预测

氧化亚铁(FeO)含量是衡量烧结矿强度和还原性的重要指标,烧结过程FeO含量的实时准确预测对于提升烧结质量、优化烧结工艺具有重要意义.然而烧结过程热状态参数缺失、过程参数波动频繁给FeO含量的高精度预测带来巨大的挑战,为此,提出一种基于知识与变权重回声状态网络融合(Fusion of data-knowledge and adaptive weight echo state network, DK-AWESN)的烧结过程FeO含量预测方法.首先,针对烧结过程热状态参数缺失的问题,建立烧结料层最高温度分布模型,实现基于料层温度分布特征的FeO含量等级划分;其次,针对烧结过程参数波动频繁的问题,提出基于核函数高维映射的多尺度数据配准方法,有效抑制离群点的影响,提升建模数据的质量;最后,针对烧结过程数据驱动模型缺乏机理认知致使模型预测精度不高的问题,将过程数据中提取得到的FeO含量等级知识与AW-ESN (Adaptive weight echo state network)结合,建立DK-AWESN模型,有效提升复杂工况下FeO含量的预测精度.现场工业数据试验表明,所提方法能实时准确地预测烧结过程FeO含量,为烧结过程的智能化调控提供实时有效的FeO含量反馈信息.

基于蝙蝠算法优化ESN的氯乙烯质量分数软测量模型预测

为解决氯乙烯因其精馏过程具有较强的非线性,无法实现对氯乙烯质量分数实时测量的问题,提出一种基于蝙蝠算法(bat algorithm,BA)优化回声状态网络(echo state network,ESN)的软测量模型BA-ESN.首先,通过对氯乙烯精馏过程的分析,选取模型的辅助变量,并将归一化处理后的数据作为模型输入变量;其次,由于回声状态网络中的权值和阈值都是随机产生的,影响其泛化能力,故采用蝙蝠算法对回声状态网络的输出权值进行优化,从而提高ESN模型的收敛速度;最后,将BA-ESN模型预测氯乙烯质量分数的预测结果与ESN模型和BP模型的预测结果进行对比.仿真结果表明:BA-ESN模型的预测精度较高,泛化能力和鲁棒性都较好,能够满足氯乙烯精馏过程实时测量的要求.

基于Echo State Neural Networks的短期交通流预测算法

在城市交通环境,交通流的正确预测是比较困难,因为多个十字路口,这使得预置的交通控制模型之间的相互作用和intertwinement不能保持始终高性能在所有的交通情况。

基于PLESN和LESQRN概率预测模型的短期电力负荷预测

针对现有电力负荷预测不能很好反映负荷数据的周期性和趋势性以及残差的波动性特征提出一种考虑周期性建模的泄露积分型回声状态网络点预测模型和泄露积分型回声状态分位数回归网络概率预测模型组合的短期电力负荷预测方法.首先为了捕捉负荷的多重特征定义了周期性和趋势性损失函数辅助优化点预测模型然后为克服残差的波动问题利用概率预测模型对点预测值与真实值的残差进行建模预测最后整合同时刻的点预测值与残差预测区间得到概率预测模型结果.实际算例结果表明与其他模型相比所提模型不仅有效抑制尖端振荡现象而且能够生成可靠的概率密度分布.

基于EMD-DESN的无人机集群航迹目的地预测

无人机(unmanned aerial vehicle,UAV)集群作战样式多样、运动模式复杂,导致集群航迹目的地难以预测。为解决上述问题,本文提出了一种基于经验模态分解(empirical mode decomposition,EMD)和深度回声状态网络(deep echo state network,DESN)的UAV集群航迹目的地预测算法。为使集群运动模型更真实地模拟UAV集群作战过程,本文引入航向误差时变方差,改进了Olfati-Saber集群运动模型的虚拟领导项。为处理因群内的协同作用和集群航向误差导致的运动非平稳性,引入了EMD,对UAV航迹序列进行重构。考虑到获知航迹的时序性,设计了滑窗结构,采用DESN对重构航迹的不同时段进行目的地预测。仿真实验结果表明,本文提出的EMD-DESN算法较基本DESN算法能以更高的准确度预测UAV集群航迹目的地,并能更早地实现稳定的正确预测。

Simplified Echo-State-Network Based Services Awareness for High-Speed Passive Optical Network

With the challenge from services diversity grows greatly,the service-oriented supporting ability is required to current high-speed passive optical network(PON) .Aimed to enhance the quality of service(Qo S) brought by diversified-services,this paper proposes an Simplified Echo State Network(SESN) Based Services Awareness scheme in High-Speed PON(Passive Optical Network) .In this proposed scheme,the ring topology is adopted in the reservoir of SESN to reduce the complexity of original Echo State Network,and system dynamics equation is introduced to keep the accuracy of SESN.According to the network architecture of 10G-EPON,a SESN Master is running in the OLT and a number of SESN Agents work in ONUs.The SESN Master plays the main function of service-awareness from the total view of various kinds services in 10G-EPON system,by fully SESN training.Then,the reservoir information of well-trained SESN in OLT will be broadcasted to all ONUs and those SESN Agents working in ONUs are allowed to conducts independent service-awareness function.Thus,resources allocation and transport policy are both determined just only in ONUs.Simulation results show that the proposed mechanism is able to better supporting ability for multiple services.

Short-term prediction of photovoltaic power generation based on LMD-EE-ESN with error correction

Considering the instability of the output power of photovoltaic(PV)generation system,to improve the power regulation ability of PV power during grid-connected operation,based on the quantitative analysis of meteorological conditions,a short-term prediction method of PV power based on LMD-EE-ESN with iterative error correction was proposed.Firstly,through the fuzzy clustering processing of meteorological conditions,taking the power curves of PV power generation in sunny,rainy or snowy,cloudy,and changeable weather as the reference,the local mean decomposition(LMD)was carried out respectively,and their energy entropy(EE)was taken as the meteorological characteristics.Then,the historical generation power series was decomposed by LMD algorithm,and the hierarchical prediction of the power curve was realized by echo state network(ESN)prediction algorithm combined with meteorological characteristics.Finally,the iterative error theory was applied to the correction of power prediction results.The analysis of the historical data in the PV power generation system shows that this method avoids the influence of meteorological conditions in the short-term prediction of PV output power,and improves the accuracy of power prediction on the condition of hierarchical prediction and iterative error correction.

Echo-state-network classification based multi-services awareness in high-speed optical passive networks

With the challenge of great growing of services diversity,service-oriented supporting ability is required by current high-speed passive optical network( PON). Aimed at enhancing the quality of service( Qo S) brought by diversified-services,this study proposes an echo state network( ESN)based multi-service awareness mechanism in 10-Gigabite ethernet passive optical network( 10GEPON). In the proposed approach,distributed architecture is adopted to realize this ESN based multi-service awareness. According to the network architecture of 10G-EPON,where a main ESN is running in OLT and a number of ESN agents works in ONUs. The main-ESN plays the main function of service-awareness from the total view of various kinds of services in 10G-EPON system,by full ESN training. Then,the reservoir information of well-trained ESN in OLT will be broadcasted to all ONUs and those ESN agents working in ONUs are allowed to conduct independent service-awareness function. Thus,resources allocation and transport policy are both determined only in ONUs. Simulation results show that the proposed mechanism is able to better support the ability of multiple services.

基于NGO-VMD-SSA-ESN的短期电价预测

针对电价波动性和非线性的特点,为提高电价预测的精度,提出了一种基于回声状态网络的短期电价混合预测模型。首先,基于北方苍鹰优化算法(NGO)优化后的变分模态分解(VMD)对原始电价进行分解,降低电价的波动性;然后,利用麻雀搜索算法(SSA)对回声状态网络(ESN)的参数进行优化,使其能针对NGO-VMD分解后的不同子序列自适应地调整参数进行预测,降低参数经验设置的随机性;最后,根据分解子序列与原始数据的皮尔逊相关系数,选择合适子序列的预测结果重构合成最终预测结果,消除了噪声的影响。以美国PJM电力市场为例,与其他电价预测模型对比验证所提出的混合模型具有更好的预测精度。

Stock Price Forecasting: An Echo State Network Approach

Forecasting stock prices using deep learning models suffers from pro-blems such as low accuracy,slow convergence,and complex network structures.This study developed an echo state network(ESN)model to mitigate such pro-blems.We compared our ESN with a long short-term memory(LSTM)network by forecasting the stock data of Kweichow Moutai,a leading enterprise in China’s liquor industry.By analyzing data for 120,240,and 300 days,we generated fore-cast data for the next 40,80,and 100 days,respectively,using both ESN and LSTM.In terms of accuracy,ESN had the unique advantage of capturing non-linear data.Mean absolute error(MAE)was used to present the accuracy results.The MAEs of the data forecast by ESN were 0.024,0.024,and 0.025,which were,respectively,0.065,0.007,and 0.009 less than those of LSTM.In terms of con-vergence,ESN has a reservoir state-space structure,which makes it perform faster than other models.Root-mean-square error(RMSE)was used to present the con-vergence time.In our experiment,the RMSEs of ESN were 0.22,0.27,and 0.26,which were,respectively,0.08,0.01,and 0.12 less than those of LSTM.In terms of network structure,ESN consists only of input,reservoir,and output spaces,making it a much simpler model than the others.The proposed ESN was found to be an effective model that,compared to others,converges faster,forecasts more accurately,and builds time-series analyses more easily.

A Prediction Method Based on Improved Echo State Network for COVID-19 Nonlinear Time Series

This paper proposes a prediction method based on improved Echo State Network for COVID-19 nonlinear time series, which improves the Echo State Network from the reservoir topology and the output weight matrix, and adopt the ABC (Artificial Bee Colony) algorithm based on crossover and crowding strategy to optimize the parameters. Finally, the proposed method is simulated and the results show that it has stronger prediction ability for COVID-19 nonlinear time series.

基于ESN模型的制造商库存需求预测

准确的库存需求预测对制造商提升库存管理能力具有重要意义。首先,针对制造商库存需求非平稳性、非线性和高波动的特点,使用奇异谱分析(SSA)和变分模态分解(VMD)实现特征工程,将原始数据分解为趋势信号和多种不同周期的信号,作为模型的输入。然后,引入回声状态网络(ESN),提出SSA-VMD-ESN模型,对库存需求进行预测。最后,将所提模型应用到Kaggle平台真实的制造商库存需求数据集中,与6种模型对比,并单独使用9个产品实验,结果表明所提出模型各项误差指标均属最优。由此证明,研究所提出模型可以有效提高库存需求预测的准确率,对库存管理具有较强的应用价值。

改进ESNs在通信话务量预测上的应用研究

现有回声状态网络(Echo State Networks,ESNs)的通信话务量预测方法只考虑了历史通信话务量对预测性能的影响,较少涉及多个输入变量的通信话务量预测问题。文中首先针对ESNs用于实际多元时间序列预测任务时训练效率低,输入数据维数较多时计算复杂度大的问题,提出用改进的交替方向乘子算法(IAD-ESNs算法)训练ESNs;针对单一输入变量不能提供更加全面的预测信息,提出了改进ESNs的多变量预测模型(MP-IADMM-ESNs)。以真实通信话务量数据进行仿真实验,结果表明,提出的预测模型MP-IADMM-ESNs对多变量通信话务量预测有较高的预测精度和预测效率。

基于ISSA-NESN的可再生能源电力需求预测研究

首先将标准麻雀搜索算法SSA加入Tent混沌映射以及动态步长因子得到优化的麻雀搜索算法ISSA,以提高种群的多样性并调节种群的全局搜索能力与局部开发能力。然后,将标准回声状态网络ESN的储蓄池内部状态函数用双曲正切函数来代替得到非线性回声状态网络NESN。最后,利用优化的麻雀搜索算法ISSA对非线性回声状态网络的储蓄池稀疏度SD以及谱半径SR进行参数优化,构建ISSA-NESN预测模型。通过算例分析,ISSA-NESN的平均绝对百分比误差(MAPE)为15.84%,均方根误差(RMSE)为0.12,预测效果优于其他对比模型。

基于VAE和DLIESN的工控系统入侵检测方法

针对现有工控系统入侵检测方法训练复杂,且对攻击样本检测率低的问题,提出一种基于变分自编码器(VAE)和深度漏积回声状态网络(DLIESN)的入侵检测方法。使用VAE对数据集中的罕见攻击类样本进行扩充,平衡样本分布;构建DLIESN分类器,其层次化的储备池结构设计使模型获得更好的动态特性,提升分类精度。通过生成数据评价实验验证了VAE生成样本的有效性,确定适合DLIESN的网络结构,分析不同漏积参数对DLIESN模型性能的影响。对比实验结果表明,DLIESN取得了最短训练耗时,其检测性较优。

基于SSA-MIC-SMBO-ESN的水质预测模型

水污染事件威胁人类生产和生活安全,提前预测水质变化对水污染的快速反应具有重要意义。基于水质数据的时序性,引入泄漏积分型回声状态网络(ESN),以莆田市东圳水库水质监测站的10种水质指标数据作为样本,分别构建DO、COD_(Mn)、TP的水质预测模型。首先,在用邻近点线性趋势法对缺失值进行填充,用Z-score法和邻近点线性趋势法对异常值进行检测修正的基础上,用奇异谱分析(SSA)算法对水质数据进行平滑降噪处理;然后,采用最大互信息系数(MIC)衡量水质指标之间的相关度,选取相关系数较大的水质指标作为待预测水质指标的输入特征;最后,利用ESN构建基于多特征的水质预测模型,其中采用序列模型优化(SMBO)算法对模型的超参数进行优化。试验结果表明,构建的DO、COD_(Mn)和TP的SSA-MIC-SMBO-ESN水质预测模型都具有较高的预测精度,适合实际应用。

融合时间序列趋势的Dual-ESN机组负荷预测模型

针对传统模型在机组负荷预测中无法充分捕获内部多变量演化模式的问题,提出了一种基于时间序列的趋势和数值信息融合的双重回声状态网络Dual-ESN(dual-echo state network)机组负荷动态预测模型。首先,引入最小二乘法,对相关的多元历史信息按照局部时间跨度进行趋势拟合。进一步,得到有关过程变化的模式序列,并和原本的数值分别被送入两个独立的储备池,以并行的时间维度进行特征学习。其次,将隐层的高维空间状态送入输出层,融合信息,得到所需要的预测结果。最后,基于山西某工厂660 MW机组装置的真实数据集,进行验证。对比已有预测方法,结果表明所提预测模型在多种性能指标上均有提升。