Fast Learning in Spiking Neural Networks by Learning Rate Adaptation

For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.

神经网络在交通图像压缩中的应用

利用神经网络的结构特征及良好的数据处理能力、在数据压缩领域拥有的先天优势,采用性能优良的BP算法构建网络模型。分析BP网络压缩的原理,搭建压缩系统模型,分别使用自适应学习率梯度下降法和BFGS拟牛顿法训练网络,从而实现交通图像的压缩与重建。实验结果表明,基于BFGS拟牛顿法的BP神经网络收敛速度快,压缩性能优良,获得低压缩率的同时重建图像视觉效果良好。

BP算法的改进及仿真研究

BP算法是神经网络中最常用的算法之一。分析传统的BP算法思想,发现它存在着收敛速度慢的问题。针对此问题,归纳一些行之有效的改进方法,本文论述这些方法的主要思想,并分析它们的改进效果,即仿真结果,结果表明改进算法的学习收敛速度大大的优于标准BP算法。