一种回声状态网络的权值初始化方法

A weight initialization method for echo state network

为了避免奇异解,提高网络性能,给出一种回声状态网络的权值初始化方法(WIESN).利用柯西不等式和线性代数确定优化的初始权值的范围与输入维数、储备池维数、输入变量和储备池状态相关,从而确保神经元的输出位于sigmoid函数的激活区域.实验结果表明,权值初始化方法的精度和训练时间要优于随机初始化方法,且相比于训练时间,权值初始化的时间是可以忽略不计的.

To avoid singular problem and improve the performance of neural network, a weight initialization method for echo state network（WIESN） is proposed. With Cauchy inequality and linear algebra, the range of optimal initial weights, which is related to input dimension, reservoir dimension, input variables and reservoir state, is determined. The proposed method ensures that the outputs of neurons are in the active region. Simulation results show that the accuracy and training time of the proposed method is better than learning with random initialization. In addition, the time for weight initialization process is negligible comparing with the training process.