结合侧抑制机制的自动编码器训练新算法

A NEW TRAINING ALGORITHM OF AUTOENCODER COMBINING LATERAL INHIBITION MECHANISM

深度学习是目前最热门的机器学习方法之一。针对深度学习中的自动编码器在训练时容易产生网络模型复杂度过高、输出矩阵不够稀疏、小样本训练过拟合等问题,提出一种结合侧抑制机制的自动编码器训练新算法。算法构建了用于隐藏层的侧抑制神经元筛选模型。首先设定抑制限寻找符合抑制条件的神经元,然后通过侧抑制函数对符合条件的神经元进行快速输出抑制,运用反向传播算法对模型进行优化,最终输出权重特征。实验结果表明,算法能够使隐藏层输出近似满足稀疏条件并学习得到更加鲁棒的特征,提高分类正确率的同时还能一定程度上抑制过拟合现象。

Deep learning has emerged as one of the most popular machine learning means. When in training, the autoencoder in deep learning is easy to produce the problems of excess network model complexity, insufficient output matrix sparsity and over-fitting in small sample training, etc. Aiming at such issues, we present a new training algorithm for autoencoder which combines the lateral inhibition mechanism. The algorithm builds the lateral inhibition neuron screening model used in hidden layer, it first sets up the inhibition threshold for seeking the neurons satisfying inhibition condition, then inhibits the fast output of neurons meeting the condition with lateral inhibition function, and employs back propagation algorithm to optimise the model and finally outputs the weight characteristics. Experimental result proves that the algorithm can make the hidden layer output approximately meet the sparse condition and get more robust characteristics through learning, while improving correctness rate of classification, it is able to restrain over-fitting phenomenon to certain extent as well.