受限玻尔兹曼机的稀疏化特征学习

Sparse Feature Learning for Restricted Boltzmann Machine

受限玻尔兹曼机(RBM)作为深度学习算法的一种基础模型被广泛应用,但传统RBM算法没有充分考虑数据的稀疏化特征学习,使得算法性能受数据集的稀疏性影响较大。提出一种RBM稀疏化特征学习方法(sRBM),通过归一化的输入数据均值确定数据集的稀疏系数,将稀疏系数大于阈值的稠密数据集自动转化为稀疏数据集,在不损失信息量的情况下实现输入数据的稀疏化。在手写字符数据集和自然图像数据集上的实验结果表明,sRBM通过输入数据稀疏化有效提升了RBM的稀疏化特征学习性能。

As a basic model for deep learning algorithms, restricted boltzmann machine （RBM） is widely applied in the field of machine learning. However, the traditional RBM algorithm does not take full account of the sparse feature learning for data. Therefore, the algorithm performance is greatly influenced by the sparsity of the dataset. In this study, a sparse feature learning method for restricted Boltzmann machine（sRBM） was proposed. Firstly, the sparse coefficient of the dataset is determined by the mean of normalized input data. Then the dense dataset with the sparse coefficient being greater than threshold will be converted to sparse dataset automatically. As a result, sRBM makes the input data sparse without information loss. We performed experiments on MNIST dataset and attribute discovery dataset. The experimental results show that sRBM improves the performance of sparse feature learning for RBM effectively.