基于惩罚函数泛化的神经网络剪枝算法研究

Study of Neural Network Pruning Algorithm Based on Generalization of Penalty Function

神经网络的隐层数和隐层节点数决定了网络规模,并对网络性能造成较大影响。在满足网络所需最少隐层节点数的前提下,利用剪枝算法删除某些冗余节点,减少隐层节点数,得到更加精简的网络结构。基于惩罚函数的剪枝算法是在目标函数后加入一个惩罚函数项,该惩罚函数项是一个变量为网络权值的函数。由于惩罚函数中的网络权值变量可以附加一个可调参数,将单一惩罚函数项泛化为一类随参数规律变化的新的惩罚函数,初始惩罚函数可看作泛化后惩罚函数的参数取定值的特殊情况。实验利用基于标准BP神经网络的XOR数据进行测试,得到隐层节点剪枝效果和网络权值随惩罚函数的泛化而发生变化,并从数据分析中得出具有更好剪枝效果及更优网络结构的惩罚函数泛化参数。

The number of hidden layer and hidden layer node in neural network determines the size of the network and has a great influence on the performance of the network. Therefore,when the network contains the least hidden layer node number,pruning algorithm can be used to delete some redundant node, then the network is more simple. The pruning algorithm adds a penalty function to the target function, and the penalty function regards the weights of network as variable. It adds a variable parameter to the weights of network,so the simple penalty function can be generalized to a kind of new penalty function that changes as the parameter. The initial function can be treated as a special condition after the generalization of penalty function. Experiment tests the XOR data based on the BP neural network and sums up the effect of the generalization of penalty function on the pruning of the hidden layer node with neural network and the structure of the neural network. Then the parameters which can lead to better pruning effect and more optimal network structure are obtained from data in experiment.