一种新的基于光滑L_0范数的受限玻尔兹曼机

A New Restricted Boltzmann Machine Based on Smooth L_0 Norm

受限玻尔兹曼机(Restricted Boltzmann machines,RBM)是一种有效的特征提取器,它是深度信念网络的基本组成模块。为了进一步提升RBM的数据表示性能,受人类大脑视觉稀疏表示启发,提出一种新的稀疏RBM,即SmoothRBM。它通过添加一个光滑L_0范数的正则项来直接约束隐层单元的总体激活概率,可以根据不同的学习任务学习到不同的稀疏水平。MNIST数据集上的相关实验表明,SmoothRBM模型与当前的一些优秀模型相比,可以更有效的提取数据集中的特征信息,学习到更稀疏和更具判别力的表示形式。

It is known that Restricted Boltzmann machines(RBM)can be used as an effective feature extractor and building blocks of the deep belief networks.To further improve the performance of data representation,this paper proposes a new sparse RBM which is inspired by the vision sparse representation of the human brain,referred to as SmoothRBM.It directly constrains the activation probability of the hidden units by adding a smooth L0 norm regularization and can learn different sparse levels according to different learning tasks.Experiments on the MNIST dataset show that the SmoothRBM can extract feature information from dataset more efficiently,learn more sparser and more discriminative representations than the related state-of-the-art models.