一种深度回声状态网络的输入尺度自适应算法

An Adaptive Algorithm of Input Scale for Deep Echo State Networks

深度回声状态网络是回声状态网络与深度学习思想的结合,合理选取不同谱半径的内部状态矩阵和弱积分参数能有效增强深度回声状态网络的多尺度时域特性。利用数据可视化分析输出矩阵在不同网络层中的分布关系,发现高层网络中部分神经元处于饱和工作状态且该状态抑制了网络动态预测能力。提出一种深度回声状态网络的输入矩阵自适应算法,在对网络内部状态的均值和方差进行递推估计的基础上判断神经元饱和状态,通过自适应调整各层输入权重的值来增强神经元动态性。数值计算结果表明,基于输入尺度自适应算法的深度回声状态网络相对同等规模的单层回声状态网络对于动态系统的预测精度有成倍提升。

Deep Echo State Networks(DESN)is a combination of Echo State Networks(ESN)and the idea of deep learning.A reasonable selection of internal state matrices and weak integration parameters with different spectral radius can effectively enhance the multi-scale time domain characteristics of the DESN.By analyzing the distribution of output matrix in different network layers through data visualization,it is found that part of the neurons in higher network layers are partially working in a saturated state,which weakens the dynamic prediction of the network.An adaptive algorithm of input matrix for DESN is proposed,based on the recursive estimation of the mean and variance of the internal network state,whether a neuron is saturated is judged.Then the output weight of each layer is adjusted adaptively to improve neuron dynamics.The numeric analysis results show that the DESN based on the input scale adaptive algorithm has doubled the prediction accuracy of the dynamic system compared with the single-layer ESN of the same scale.