基于SCAD-ESN的时间序列预测模型

Time-series Prediction Model Based on SCAD-ESN

回声状态网络(ESN)是一种重要的时间序列预测方法,但在训练数据存在噪声或野点情况下,ESN将会出现过拟合问题。针对该问题,提出基于平滑消边绝对偏离罚函数的回声状态网络(SCAD-ESN)模型。不同于在模型中加入岭回归、L1范数罚函数及小波降噪等常规方法,该模型利用SCAD罚函数对变量进行选择,将小变量置为零以满足变量稀疏性,将大变量直接置为常数,从而能够很好地解决ESN过拟合问题并满足近似无偏估计。对于SCAD罚函数的非凸函数优化问题,提出基于局部二次近似(LQA)的求解方法,将最小角回归(LQR)方法用于SCAD罚函数求解,避免了计算量巨大的问题。使用基于粒子群优化(PSO)的超参数选取方法快速确定平滑消边绝对偏离–回声状态网络模型的超参数,克服利用经验选取超参数时存在的盲目性较大且难以确定整体最优的超参数问题。混沌系统数值仿真和网络流量仿真结果表明,相对于常规模型,该模型能有效地降低测试误差,从而克服过拟合问题。

Echo state network （ESN） is an important method for time series prediction.However,the overfitting problem is likely to occur when the training data contain noise or outliers.To solve this problem,an ESN model based on smoothly clipped absolute deviation （SCAD） penalty func- tion was proposed in this paper.Different from the traditional methods,such as ridge regression,L1 norm penalty,wavelet denoising and other meth- ods added into the ESN model,the SCAD penalty function was used to select the variables of the ESN model.Specially, to meet the variable sparse- ness,the small coefficients are set to zero.And the large coefficients are taken as constants,which can well solve the over-fitting problem of ESN and satisfy approximate unbiased estimation.For the nonconvex optimization problem of SCAD penalty function,the local quadratic approxima- tion （LQA） solution was presented in the paper, and the enormous computational complexity of the least angle regression （LQR） method for solv-~ ing the SCAD penalty function was overcome.Then,the particle swarm optimization （PSO） is used to quickly determine the hyperparameters se- lection of smoothly clipped absolute deviation-echo state network （SCAD-ESN） model.The proposed method overcame the blindness of the con- ventional methods using the experience to select the hyperparameters,which is blind and difficult to determine the global optimum.Finally,the chaotic system simulation and network traffic simulation showed that,compared with the conventional models,the model can effectively reduce the test error and overcome overfitting problem.