退火期望最大化算法A-EM

An Annealing Expectation Maximization Algorithm

使用EM算法训练随机多层前馈网具有低开销、易于实现和全局收敛的特点,在EM算法的基础上提出了一种训练随机多层前馈网络的新方法AEM.AEM算法利用热力学系统的最大熵原理计算网络中隐变量的条件概率,借鉴退火过程,引入温度参数,减小了初始参数值对最终结果的影响.该算法既保持了原EM算法的优点,又有利于训练结果收敛到全局极小.从数学角度证明了该算法的收敛性,同时,实验也证明了该算法的正确性和有效性.

Training the stochastic feedforward neural network with expectation maximization （EM） algorithm has many merits such as reliable global convergence, low cost per iteration and easy programming. A new algorithm named A-EM （annealing-expectation maximization） based on the EM algorithm is proposed for training the stochastic feedforward neural network. The A-EM algorithm computes the condition probability of the hidden variable in the network system through the maximum entropy principle of the thermodynamics. It can reduce the influence of the initial value on the final resolution by simulating the annealing process and introducing the temperature parameter. This algorithm can not only keep the merits of the original EM, but also facilitate the results converge to the global minimum. The convergence of the algorithm is proved and its correctness and validity is verified by experiments.