基于动量方法的受限玻尔兹曼机的一种有效算法

An Effective Algorithm of Restricted Boltzmann Machine Based on Momentum Method

深度学习给模式识别与机器学习带来了巨大的变化,已成功应用于语言处理、图像处理、信号处理、商业经济等方面.受限玻尔兹曼机(Restricted Boltzmann Machine,RBM)是一个表示能力强、很好的生成模型,多个RBM堆叠而构成的深度信念网络模型(Deep Belief Nets,DBN)的学习时间会较长.为加快整个DBN网络的学习时间和提高分类效果,本文提出基于动量方法 RBM的一种有效算法.该算法在RBM预训练阶段,结合梯度上升算法特点采取快速上升的动量方式;以及BP算法微调阶段,为了能精确的找到最优点,结合梯度下降算法特点,相应的引入缓慢下降式的动量项,即在梯度上升和梯度下降过程中都使用不同的动量方式.本文算法在MNIST手写数字体和CMU-PIE人脸数据库上进行了实验,结果表明,提出的改进算法能够有效地增强图像特征的表达能力,提高图像的分类效果和实验效率.

Deep learning is bringing revolution to pattern recognition and machine learning,which has been successfully applied to language processing,image processing,signal processing,business economy and so on.Restricted Boltzmann machine (RBM) is a strong representation and generative mod el,however,the learning time of deep belief nets (DBN),which consists of multiple stacking RBM,will be longer.In this paper,the improved momentum method is used not only in gradient ascent algorithm but also in gradient descent algorithm for both classification accuracy enhancement and training time decreasing.According to the characteristics of the gradient ascent algorithm,a rapidly ascending momentum method is used in the RBM pre-training phase,which greatly improves the speed of learning.According to the characteristics of the gradient descent algorithm,an improved slowly descending momentum term is also used in the fine-tuning stage to accurately find the best point.Through the recognition experiments on the MNIST dataset and CMU-PIE face dataset,the achieved results show that the improved momentum algorithm can effectively enhance the ability of image feature expression and improve both accuracy and computation efficiency.