基于MIC的深度置信网络研究

Deep Belief Networks Research Based on Maximum Information Coefficient

传统的深度置信网络(DBNs)训练过程采用重构误差作为RBM网络的评价指标,它能在一定程度上反映网络对训练样本的似然度,但它并不是可靠的。而最大信息系数(MIC)能反映两个属性间的相关度,保留相关度较大的属性,且MIC较稳健,不易受异常值的影响,可作为网络评价指标。故提出一种基于最大信息系数(MIC)的深度置信网络方法,一方面用MIC对数据进行降维预处理,提高数据与网络的拟合度,降低网络分类误差;另一方面将MIC作为网络评价标准,改进重构误差的不可靠性。分别利用传统方法与基于MIC的深度置信网络方法对手写数据集MNIST和USPS进行分类实验,结果表明,基于MIC的深度置信网络方法能有效地提高识别率。

The traditional deep belief networks use reconstruction error as the evaluation criteria of restricted boltzmann machine（RBM） networks in the training process, which can reflect the likelihood between RBM network and training samples to some extent. However, it is not reliable. Maximum information coefficient （MIC）, based on the estimations of Shannon entropy and conditional entropy, identifies interesting relationships between pairs of variables in large data sets and captures a subset of highly related features. The MIC can be used as a criterion for evaluating a network since it is robust to outliers. In order to construct models that fit data well and reduce classification error, a deep belief net- works based on MIC method was proposed. MIC is used not only in dimensionality reduction, but also in improving the unreliability of the reconstruction error. Classification experiments were performed on handwriting data sets of MNIST and USPS by several traditional methods and deep belief networks based on MIC method. The results show that the lat- ter can improve the recognition rate effectively.