深度学习最佳深度的确定

Determination of the optimum depth in deep learning

确定最佳深度可以降低运算成本,同时可以进一步提高精度。针对深度置信网络深度选择的问题,文章分析了通过设定阈值方法选择最佳深度的不足之处。从信息论的角度,验证了信息熵在每层玻尔兹曼机(RBM)训练达到稳态之后会达到收敛,以收敛之后的信息熵作为判断最佳层数的标准。通过手写数字识别的实验发现该方法可以作为最佳层数的判断标准。

Abstract：The best depth can reduce operation cost while can improve the precision . Aiming at the problem of deep b elief networls dep th selec-tion , the paper analyzes the deficiencies of setting the threshold value metliod to select the b est depth . From the perspective of in formation theo-ry ,firstly , by verifying the in formation en tropy will converge in each layer of the Boltzmann machine （ RBM ） after training reaching steady state, convergence of in formation entropy as criteria for ju d ging the best layers. T h rough the experiments can be found that this meth iod can be used as a criterion of the optimal layer .