ANN构造性设计中基于GA优选神经元激活函数类型

Optimizing Neural Activation Function Types Based on GA in Con structive ANN Design

构造性设计是ANN设计的发展方向之一。全面的高质量的ANN学习应包括神经元激活函数类型的自动优化。该文在构造性设计的框架内讨论了如何实现典型前馈网络的包括神经元激活函数类型在内的全面学习。首先,提出了典型前馈网络的一种构造性设计方法的原理和算法框架,把整个网络的设计分解成了一个个单个神经元的设计问题;然后提出了基于GA的能实现激活函数类型优选的单个神经元的设计方法。大量函数拟合的仿真实验显示:与其它几种激活函数类型不优选的常见ANN设计方法相比,该文提出的方法更有效,能用较小的网络结构获得较好的泛化性能。

Constructive algorithms will be preferable in ANN designing,and perfect ANN trainings should have the auto-optimization of neural activation function types.In this paper,aiming at a typical feedforward neural network(FNN),a constructive ANN designing algorithm with perfect training,i.e.,including the auto-optimization of neural activation function types,is investigated.First,the principle and the frame of the constructive algorithm are given,in which the design of the whole FNN is changed into the designs of single neurons one by one.Second,based on genetic algorithms ,the design algorithm of a single neuron is proposed,in which the suitable neural activation function type is chosen automatically from a base.Finally,with lots of function approximation experiments,it is shown that,compared with some other ANN designing algorithms without activation function type optimization,the proposed algorithm in this paper is more feasible,achieving better ANN generalization with smaller network size.