基于贝叶斯ELM的隐层节点数稀疏性研究

Sparsity of Hidden Layer Nodes Based on Bayesian Extreme Learning Machine

在贝叶斯极限学习机中,模型隐层节点数的确定尚无理论依据,一种基于多响应稀疏回归方法(Multiresponse Sparse Regression,MRSR)的隐层节点数稀疏方法由此被提出。首先根据传统神经网络确定隐层节点数的经验方法设定模型的隐层节点数为区间范围的上限,然后利用MRSR方法对模型输出矩阵和标签向量进行稀疏性回归分析求解,最后能得到一个既能对隐层节点数进行稀疏也能对样本个数进行稀疏的贝叶斯极限学习机模型。仿真结果表明该方法在满足精度要求的前提下能剔除冗余的隐层节点,进一步实现了模型的稀疏性。

Considering the fact that there is no theory for the determination of hidden layer nodes in Bayesian extreme learning machine, this paper proposes a method of sparse hidden layer nodes based on multiresponse sparse regression. First, the hidden layer nodes are set to the upper limit of the interval range determined by the traditional experience method. Then the sparse regression between the model output matrix and the target vector is solved by MRSR. At last, we can get a sparse model for both the hidden layer nodes and training samples. The simulation results show that the method can prune the redundant nodes without sacrificing the model accuracy, which further achieves model sparsity.