多层感知机权值扰动敏感性计算算法研究

Research on Computation Algotithm of Multilayer Perceptron Sensitivity to Weights Perturbation

多层感知机(MLP)对权值扰动的敏感性反映当网络权值发生变化时网络输出的变化规律,是研究MLP学习机制的一种重要衡量工具。系统讨论MLP对权值扰动的敏感性计算方法,提出一种层层递进的敏感性近似算法。利用数值积分从第一层神经元开始计算,后一层神经元的计算利用前一层的结果,最终给出所有层神经元以及整个网络的敏感性计算表达式。该敏感性计算算法只要求网络的各维输入相互独立,而对其具体的分布无任何限制,同时还具有计算复杂度低、通用性强等优点,模拟实验验证了该算法的准确性和有效性。

Multilayer Perceptron （MLP） sensitivity to weights perturbation reflects the change rule of the network output when network weights get disturbed. It is an important measure for research on learning mechanism. This paper systematically discusses the computation of the sensitivity to weights perturbation and proposes an approximation algorithm which computes the sensitivities layer by layer. The sensitivity is defined as the mathematical expectation of absolute output deviations due to weights perturbation. Firstly the algorithm computes the sensitivities of the first layer and then the next layer which will use the former layer＇ s results and finally gives the computation expressions of all the neurons and the whole network. This algorithm requests a weak assumption on the input,that is,input elements need only to be independent of each other. It also has low computation complexity and is more applicable to real applications. Experimental results demonstrate the proposed algorithm is highly accurate.