基于多新息理论的深度信念网络算法

Deep belief network algorithm based on multi-innovation theory

针对深度信念网络(DBN)算法在采用反向传播修正网络的连接权值和偏置的过程中,容易产生梯度小、学习率低、误差收敛速度慢等问题,提出一种结合多新息理论对标准DBN算法进行改进的算法,即多新息DBN(MIDBN)。MI-DBN算法是对标准DBN算法中反向传播的过程重新建模,使得算法在原先只利用单个新息的情况下,扩展为能够充分利用之前多个周期的新息,从而大幅提高误差收敛速度。通过实验对MI-DBN算法和其他分类算法进行了数据集分类的比较,实验结果表明,MI-DBN算法相较其他分类算法,其误差收敛速度较快,而且最终对MNIST数据集和Caltech101数据集的识别中误差结果相对更小。

Aiming at the problem of small gradient, low learning rate, slow convergence of error during the process of using Deep Belief Network （DBN） algorithm to correct connection weight and bias of network by the method of back propagation, a new algorithm called Multi-Innovation DBN （MI-DBN） was proposed based on combination of standard DBN algorithm with multi-innovation theory. The back propagation process in standard DBN algorithm was remodeled to make full use of multiple innovations in previous cycles, while the original algorithm can only use single innovation. Thus, the convergence rate of error was significantly increased. MI-DBN algorithm and other representative classifiers were compared through experiments of datasets classification. Experimental results show that MI-DBN algorithm has a faster convergence rate than other sorting algorithms; especially when identifying MNIST and Caltech101 dataset, MI-DBN algorithm has the fewest inaccuracies among all the algorithms.