一种提高受限玻尔兹曼机性能的反正切函数逼近L_0范数方法

Enhancing Performance of Restricted Boltzmann Machine Using Arctan Approximation of L_0 Norm

受限玻尔兹曼机(RBMs)常常作为深度置信网络(DBN)的基本构成模块,通过训练几个RBM,DBN能够快速地被训练好以获得好的工作效果.为了获得更好的数据表示,受稀疏编码理论的启发,本文提出一种新的稀疏RBM,称为AtanRBM.与稀疏RBM(sparse RBM)不同的是,AtanRBM是添加一个arctan正则项(arctan函数逼近L_0范数)直接地约束隐含单元的概率密度空间来达到隐含单元稀疏的效果,而不是约束隐含单元的平均激活概率期望达到相同的较低稀疏水平.在MNIST数据集的实验表明,AtanRBM比当前相关的模型可以学到更稀疏和更具辨别力的表示形式或表示方法,进而由AtanRBM预训练的深层网络能够获得更好的分类效果.

Restricted Boltzmann machines { RBMs） are often cited as building blocks of a deep belief network （DBN）. By training several RBMs,DBN can be trained quickly to achieve good performance on various machine learning tasks. To further improve the performance of data representation,inspired by sparse coding theory, we propose a novel sparse RBM model in this paper,referred to as AtanRBM. Different from sparse RBM, we encourage the hidden units to be sparse through adding an arctan norm （ arctan approxi- mation of L0 norm } constraint on the probability density space of hidden units directly, rather then constraining the expected activation of every hidden unit to the same low level of sparsity. Some experiments conducted on MNIST dataset show that AtanRBM learns sparser and more discriminative representations compared with the related state-of-the-art models, and then the deep belief network can achieve better classification performance by using layer-wise unsupervised pre-training of two AtanRBMs.