基于灵敏度分析的模块化回声状态网络修剪算法

Pruning Algorithm for Modular Echo State Network Based on Sensitivity Analysis

针对回声状态网络(Echo state network, ESN)的结构设计问题,提出基于灵敏度分析的模块化回声状态网络修剪算法(Pruning algorithm for modular echo state network, PMESN).该网络由相互独立的子储备池模块构成.首先利用矩阵的奇异值分解(Singular value decomposition, SVD)构造子储备池模块的权值矩阵,并利用分块对角阵原理生成储备池.然后利用子储备池模块输出和相应的输出层权值向量,定义学习残差对于子储备池模块的灵敏度以及网络规模适应度.利用灵敏度大小判断子储备池模块的贡献度,并根据网络规模适应度确定子储备池模块的个数,删除灵敏度低的子模块.在网络的修剪过程中,不需要缩放权值就可以保证网络的回声状态特性.实验结果说明,所提出的算法有效解决了ESN的网络结构设计问题,基本能够确定与样本数据相匹配的网络规模,具有较好的泛化能力和鲁棒性.

To design the structure of echo state network(ESN), a pruning algorithm for modular echo state network(PMESN) based on sensitivity analysis is proposed in this paper. The reservoir of PMESN is made up of independent sub-reservoir modular networks. The weight matrices of sub-reservoir modular networks are obtained by the singular value decomposition(SVD), and the reservoir is generated by the block diagonal matrix theory. The residual error’s sensitivities to the sub-reservoir modular networks are defined by their outputs and weight vectors connecting to the output layer. The significance of the sub-reservoir modular networks is determined by sensitivity. The model scale adaptability is obtained and the sub-reservoir modular networks are sorted by the defined sensitivities. Then, the number of requisite sub-reservoir modular networks is calculated by the model scale adaptability. The redundant sub-reservoir modular networks with smaller sensitivities are deleted. In the pruning process, the echo state property can be guaranteed without posterior scaling of the weights. The simulation results show that the proposed method can design the compact structure of ESN effectively and has better generalization ability and robustness.