使用最小二乘法减少神经网络的隐单元

Reducing the Hidden Units in Neural Networks by Using Least Square Method

提出新的逐步减少神经元个数并保持神经网络性能的方法 .每一步中利用提出的规则之一选择被消去的单元 ,然后求解一个线性最小二乘问题调整网络中部分剩余权值 ,使简化网络的输入 -输出关系在训练集上尽量保持不变 .该方法可以得到比已有的启发式方法规模更小 ,但性能相近的网络 。

A novel pruning algorithm, which can keep the performance of the network while its neurons are removed one by one, is proposed. In each step, a hidden unit is chosen to be deleted according to one of two proposed rules, then a linear least square problem is solved to adjust part of the remaining weights in order that the performance of the reduced network is as close as possible to the original one. Compared with the existing pruning algorithms, the proposed method may lead to networks with smaller size. The simulation results of finding the functional relationship between GDP(gross domestic product) and GE(gross export), GI(gross import) show the effectiveness of the proposed method.