信息理论框架下的神经网络构建

Setting Up Neural Network Based on Framework of Information Theory

神经网络模型的构建方法是神经网络研究的重点和难点,传统的构建方法建立在实验和重复学习的基础上,本文提出了一种信息理论框架下的神经网络构建方法基于熵的神经网络(EBNN).EBNN借助于前馈网络与决策树的等价性,采用熵做为神经网络构造的准则,利用决策树的构造思想和方法,建立了一种系统的神经网络构造方法.实验表明EBNN方法学习速度比传统BP网络快,但又不降低神经网络性能.

Setting up method of the neural network model is an important and difficult problem in the research of neural network. The traditional methods were established mainly by trial and error. This paper presents a new method named Entropy-Based Neural Network, in the framework of information theory. Entropy-Based Neural Network, is a systematic setting up method of neural network by adopting entropy as the setting up criterion and utilizing the idea and techniques of decision tree under the principle of equivalence between decision tree and neural network. Experiments indicate that Entropy-Based Neural Network has a better learning speed than the traditional BP without losing the capability of the network.