DSNNs:learning transfer from deep neural networks to spiking neural networks

Deep neural networks(DNNs)have drawn great attention as they perform the state-of-the-art results on many tasks.Compared to DNNs,spiking neural networks(SNNs),which are considered as the new generation of neural networks,fail to achieve comparable performance especially on tasks with large problem sizes.Many previous work tried to close the gap between DNNs and SNNs but used small networks on simple tasks.This work proposes a simple but effective way to construct deep spiking neural networks(DSNNs)by transferring the learned ability of DNNs to SNNs.DSNNs achieve comparable accuracy on large networks and complex datasets.

Advances in memristor based artificial neuron fabrication-materials,models,and applications

Spiking neural network(SNN),widely known as the third-generation neural network,has been frequently investigated due to its excellent spatiotemporal information processing capability,high biological plausibility,and low energy consumption characteristics.Analogous to the working mechanism of human brain,the SNN system transmits information through the spiking action of neurons.Therefore,artificial neurons are critical building blocks for constructing SNN in hardware.Memristors are drawing growing attention due to low consumption,high speed,and nonlinearity characteristics,which are recently introduced to mimic the functions of biological neurons.Researchers have proposed multifarious memristive materials including organic materials,inorganic materials,or even two-dimensional materials.Taking advantage of the unique electrical behavior of these materials,several neuron models are successfully implemented,such as Hodgkin–Huxley model,leaky integrate-and-fire model and integrate-and-fire model.In this review,the recent reports of artificial neurons based on memristive devices are discussed.In addition,we highlight the models and applications through combining artificial neuronal devices with sensors or other electronic devices.Finally,the future challenges and outlooks of memristor-based artificial neurons are discussed,and the development of hardware implementation of brain-like intelligence system based on SNN is also prospected.

Advances in neuromorphic computing:Expanding horizons for AI development through novel artificial neurons and in-sensor computing

AI development has brought great success to upgrading the information age.At the same time,the large-scale artificial neural network for building AI systems is thirsty for computing power,which is barely satisfied by the conventional computing hardware.In the post-Moore era,the increase in computing power brought about by the size reduction of CMOS in very large-scale integrated circuits(VLSIC)is challenging to meet the growing demand for AI computing power.To address the issue,technical approaches like neuromorphic computing attract great attention because of their feature of breaking Von-Neumann architecture,and dealing with AI algorithms much more parallelly and energy efficiently.Inspired by the human neural network architecture,neuromorphic computing hardware is brought to life based on novel artificial neurons constructed by new materials or devices.Although it is relatively difficult to deploy a training process in the neuromorphic architecture like spiking neural network(SNN),the development in this field has incubated promising technologies like in-sensor computing,which brings new opportunities for multidisciplinary research,including the field of optoelectronic materials and devices,artificial neural networks,and microelectronics integration technology.The vision chips based on the architectures could reduce unnecessary data transfer and realize fast and energy-efficient visual cognitive processing.This paper reviews firstly the architectures and algorithms of SNN,and artificial neuron devices supporting neuromorphic computing,then the recent progress of in-sensor computing vision chips,which all will promote the development of AI.

The Lightweight Edge-Side Fault Diagnosis Approach Based on Spiking Neural Network

Network fault diagnosis methods play a vital role in maintaining network service quality and enhancing user experience as an integral component of intelligent network management.Considering the unique characteristics of edge networks,such as limited resources,complex network faults,and the need for high real-time performance,enhancing and optimizing existing network fault diagnosis methods is necessary.Therefore,this paper proposes the lightweight edge-side fault diagnosis approach based on a spiking neural network(LSNN).Firstly,we use the Izhikevich neurons model to replace the Leaky Integrate and Fire(LIF)neurons model in the LSNN model.Izhikevich neurons inherit the simplicity of LIF neurons but also possess richer behavioral characteristics and flexibility to handle diverse data inputs.Inspired by Fast Spiking Interneurons(FSIs)with a high-frequency firing pattern,we use the parameters of FSIs.Secondly,inspired by the connection mode based on spiking dynamics in the basal ganglia(BG)area of the brain,we propose the pruning approach based on the FSIs of the BG in LSNN to improve computational efficiency and reduce the demand for computing resources and energy consumption.Furthermore,we propose a multiple iterative Dynamic Spike Timing Dependent Plasticity(DSTDP)algorithm to enhance the accuracy of the LSNN model.Experiments on two server fault datasets demonstrate significant precision,recall,and F1 improvements across three diagnosis dimensions.Simultaneously,lightweight indicators such as Params and FLOPs significantly reduced,showcasing the LSNN’s advanced performance and model efficiency.To conclude,experiment results on a pair of datasets indicate that the LSNN model surpasses traditional models and achieves cutting-edge outcomes in network fault diagnosis tasks.

Spiking sychronization regulated by noise in three types of Hodgkin-Huxley neuronal networks

In this paper,we study spiking synchronization in three different types of Hodgkin-Huxley neuronal networks,which are the small-world,regular,and random neuronal networks.All the neurons are subjected to subthreshold stimulus and external noise.It is found that in each of all the neuronal networks there is an optimal strength of noise to induce the maximal spiking synchronization.We further demonstrate that in each of the neuronal networks there is a range of synaptic conductance to induce the effect that an optimal strength of noise maximizes the spiking synchronization.Only when the magnitude of the synaptic conductance is moderate,will the effect be considerable.However,if the synaptic conductance is small or large,the effect vanishes.As the connections between neurons increase,the synaptic conductance to maximize the effect decreases.Therefore,we show quantitatively that the noise-induced maximal synchronization in the Hodgkin-Huxley neuronal network is a general effect,regardless of the specific type of neuronal network.

基于灰度纹理特征提取和CS-SNN的双初级永磁同步直线电机退磁故障诊断研究

引入一种基于图像形态学纹理特征提取与布谷鸟搜索优化脉冲神经网络(cuckoo search-spiking neural network,CS-SNN)算法相结合的方法,以解决双初级永磁同步直线电机(dual primary permanent magnet synchronous linear motor,DPPMSLM)退磁故障精细定量化诊断识别的问题。首先,根据DPPMSLM拓扑结构约束,通过有限元仿真提取电机气隙空间中三线磁密信号作为有效故障信号;其次,引入图像纹理分析的方法,将一维数据信号映射为二维灰度图像,再采用伽马矫正和边缘提取技术增强图像信息,以提取图像纹理特征组成故障特征向量;然后建立两级CS-SNN分类器实现退磁故障位置类型和严重程度的精确诊断分类;最后,通过退磁样机制作和实验平台验证,提出的新方法能够准确识别DPPMSLM退磁故障位置和严重程度,并具有良好的鲁棒性,是一种有效可行的方法。

Plasticity-induced characteristic changes of pattern dynamics and the related phase transitions in small-world neuronal networks

Phase transitions widely exist in nature and occur when some control parameters are changed. In neural systems, their macroscopic states are represented by the activity states of neuron populations, and phase transitions between different activity states are closely related to corresponding functions in the brain. In particular, phase transitions to some rhythmic synchronous firing states play significant roles on diverse brain functions and disfunctions, such as encoding rhythmical external stimuli, epileptic seizure, etc. However, in previous studies, phase transitions in neuronal networks are almost driven by network parameters （e.g., external stimuli）, and there has been no investigation about the transitions between typical activity states of neuronal networks in a self-organized way by applying plastic connection weights. In this paper, we discuss phase transitions in electrically coupled and lattice-based small-world neuronal networks （LBSW networks） under spike-timing-dependent plasticity （STDP）. By applying STDP on all electrical synapses, various known and novel phase transitions could emerge in LBSW networks, particularly, the phenomenon of self-organized phase transitions （SOPTs）： repeated transitions between synchronous and asynchronous firing states. We further explore the mechanics generating SOPTs on the basis of synaptic weight dynamics.

Photonic integrated neuro-synaptic core for convolutional spiking neural network

Neuromorphic photonic computing has emerged as a competitive computing paradigm to overcome the bottlenecks of the von-Neumann architecture.Linear weighting and nonlinear spike activation are two fundamental functions of a photonic spiking neural network(PSNN).However,they are separately implemented with different photonic materials and devices,hindering the large-scale integration of PSNN.Here,we propose,fabricate and experimentally demonstrate a photonic neuro-synaptic chip enabling the simultaneous implementation of linear weighting and nonlinear spike activation based on a distributed feedback(DFB)laser with a saturable absorber(DFB-SA).A prototypical system is experimentally constructed to demonstrate the parallel weighted function and nonlinear spike activation.Furthermore,a fourchannel DFB-SA laser array is fabricated for realizing matrix convolution of a spiking convolutional neural network,achieving a recognition accuracy of 87%for the MNIST dataset.The fabricated neuro-synaptic chip offers a fundamental building block to construct the large-scale integrated PSNN chip.

Memristor-based multi-synaptic spiking neuron circuit for spiking neural network

Spiking neural networks(SNNs) are widely used in many fields because they work closer to biological neurons.However,due to its computational complexity,many SNNs implementations are limited to computer programs.First,this paper proposes a multi-synaptic circuit(MSC) based on memristor,which realizes the multi-synapse connection between neurons and the multi-delay transmission of pulse signals.The synapse circuit participates in the calculation of the network while transmitting the pulse signal,and completes the complex calculations on the software with hardware.Secondly,a new spiking neuron circuit based on the leaky integrate-and-fire(LIF) model is designed in this paper.The amplitude and width of the pulse emitted by the spiking neuron circuit can be adjusted as required.The combination of spiking neuron circuit and MSC forms the multi-synaptic spiking neuron(MSSN).The MSSN was simulated in PSPICE and the expected result was obtained,which verified the feasibility of the circuit.Finally,a small SNN was designed based on the mathematical model of MSSN.After the SNN is trained and optimized,it obtains a good accuracy in the classification of the IRIS-dataset,which verifies the practicability of the design in the network.

Effects of internal noise on the spiking regularity of a clustered Hodgkin–Huxley neuronal network

Spiking regularity in a clustered Hodgkin–Huxley（HH） neuronal network has been studied in this letter. A stochastic HH neuronal model with channel blocks has been applied as local neuronal model. Effects of the internal channel noise on the spiking regularity are discussed by changing the membrane patch size. We find that when there is no channel blocks in potassium channels, there exist some intermediate membrane patch sizes at which the spiking regularity could reach to a higher level. Spiking regularity increases with the membrane patch size when sodium channels are not blocked. Namely, depending on different channel blocking states, internal channel noise tuned by membrane patch size could have different influence on the spiking regularity of neuronal networks.

基于FPGA的移动机器人SNNs走廊场景分类器

神经形态芯片是类脑计算的重要研究内容之一,神经网络的硬件实现是神经形态芯片实现的基础。具有生物似真性的脉冲神经网络(Spiking Neural Networks, SNNs),通过尖脉冲(Spikes)传递时空信息,更适于用硬件实现,是实现类脑计算的主要工具之一。该文提出一种基于FPGA的移动机器人SNNs走廊场景分类器:将移动机器人超声传感器信息进行脉冲编码后输入到SNNs走廊场景分类器中,通过FPGA分类器的脉冲输出模式来判断机器人所处的走廊场景,从而提高机器人的环境感知能力和自主性。详细讨论了脉冲积分点火神经元模型的FPGA实现原理,以及基于此神经元模型的SNNs走廊场景分类器的硬件实现方案,仿真及实验结果证明了所提基于FPGA的移动机器人SNNs走廊场景分类器的有效性。所提走廊场景分类器不受光照条件的影响,需要的传感器测量信息少,FPGA硬件资源占有率低(LE的利用率仅10%),分类速度快、准确率高,适于实际应用。该研究不仅可以提高移动机器人的环境感知能力和自主性,而且为硬件实现SNNs提供了有益参考。

基于联合权重超图划分的SNN负载均衡方法

大规模脉冲神经网络并行模拟是探究大脑机能的重要手段。其难点在于合理地将负载映射到并行分布式平台上,提升模拟速度。为解决该问题,提出一种基于联合权重超图划分的SNN负载均衡方法,解决并行计算中进程间计算负载与通信负载的均衡问题,提高SNN模拟速度,并使用稀疏通信的方式替代集体通信,解决事件通信过程中的数据冗余问题,提升通信效率。实验结果表明,该方法使带有STDP突触20%规模的皮质层微电路模型的模拟时间,比标准循环分配算法缩短约64.5%,比普通超图分配算法缩短约57.4%,同时事件通信数据量减少了90%以上。

自校准首脉冲时间编码神经元模型

由于脉冲神经元具有复杂的时空动力过程且脉冲信息不可导,脉冲神经网络(SNN)的训练一直是一个难题。基于人工神经网络(ANN)转SNN间接训练深度SNN的方法,避免了直接训练深度SNN的难题,但该方法所获得的SNN的性能在很大程度上会受到脉冲信息编码机制的影响。在众多编码机制中,首脉冲时间编码(TTFS)具有良好的生物学基础和更高的能效,但现有TTFS编码采用单脉冲形式,信息表征能力较弱,编码所需时间窗较大。为此,在TTFS的单脉冲编码基础上,增加一个校准脉冲,形成一种自校准首脉冲时间(SC-TTFS)编码机制,并构建相应的SC-TTFS神经元模型。在SC-TTFS中,首脉冲为必定发放的脉冲,而校准脉冲根据首脉冲发放后剩余的膜电位来确定是否发放,用于对编码脉冲所引起的转换量化误差和截断误差进行补偿,同时缩小编码所需的时间窗。通过对多种编码对应的转换误差进行对比分析,以及在多种网络结构上进行ANN-SNN转换实验,验证了所提方法的优越性。采用CIFAR10和CIFAR100数据集,基于VGG和ResNet两种网络结构进行了实验验证。结果表明,所提方法在两类网络结构和两种数据集上均实现了精度无损的ANN-SNN转换,且相较于最先进的同类方法,所提方法所构建的SNN具有最短的网络推理延迟。另外,在VGG结构上,所提方法相比TTFS编码能源效率提升了约80%。

基于软阈值降噪的脉冲卷积神经网络轴承故障诊断方法

针对工业场景下滚动轴承信号易受噪声干扰,导致故障诊断准确率低和稳定性差的问题,本文提出一种基于软阈值降噪的脉冲卷积神经网络诊断方法。该方法使用软阈值滤波去噪,运用带时间标签的卷积层处理二维信号,增强动态特征提取能力。同时,通过引入IF和LIF神经元实现对时域和频域信息的联合编码,并采用替代梯度法进行端到端训练。实验结果显示,在信噪比为6dB时,所提方法的诊断准确率达100%,在信噪比为-6dB时诊断准确率达77.33%,优于其他常用方法,表明所提方法在噪声下具有良好的诊断效果和稳定性。

用于双阈值脉冲神经网络的改进自适应阈值算法

脉冲神经网络(spiking neural network, SNN)由于在神经形态芯片上低功耗和高速计算的独特性质而受到广泛的关注。深度神经网络(deep neural network, DNN)到SNN的转换方法是有效的脉冲神经网络训练方法之一,然而从DNN到SNN的转换过程中存在近似误差,转换后的SNN在短时间步长下遭受严重的性能退化。通过对转换过程中的误差进行详细分析,将其分解为量化和裁剪误差以及不均匀误差,提出了一种改进SNN阈值平衡的自适应阈值算法。通过使用最小化均方误差(MMSE)更好地平衡量化误差和裁剪误差;此外,基于IF神经元模型引入了双阈值记忆机制,有效解决了不均匀误差。实验结果表明,改进算法在CIFAR-10、CIFAR-100数据集以及MIT-BIH心律失常数据库上取得了很好的性能,对于CIFAR10数据集,仅用16个时间步长就实现了93.22%的高精度,验证了算法的有效性。

基于脉冲序列标识的深度脉冲神经网络时空反向传播算法

尖峰放电的脉冲神经网络(SNN)具有接近大脑皮层的信号处理模式,被认为是实现大脑启发计算的重要途径。但是,目前对于深度脉冲神经网络的学习仍缺乏有效的监督学习算法。受尖峰放电速率标识的时空反向传播算法的启发,该文提出一种面向深度脉冲神经网络训练的基于时间脉冲序列标识的监督学习算法,通过定义突触后电位和膜电位反传迭代因子分别分析脉冲神经元的空间和时间依赖关系,使用替代梯度的方法解决反传过程中不连续可微的问题。不同于现有基于尖峰放电速率标识的学习算法,该算法能够充分反映脉冲神经网络输出的时间脉冲序列的动态特性。因此,所提算法非常适合应用于需要较长时间序列标识的计算任务,例如行为的时间脉冲序列控制。该文在静态图像数据集CIFAR10和神经形态数据集NMNIST上验证了所提算法的有效性,在所有这些数据集上都显示出良好的性能,这有助于进一步研究基于时间脉冲序列应用的大脑启发计算。

面向神经形态感知的人工脉冲神经元的研究进展

近年来,随着人工智能技术和脉冲神经网络(SNN)的迅猛发展,人工脉冲神经元的研究逐渐兴起。人工脉冲神经元的研究对于开发具有人类智能水平的机器人、实现自主学习和自适应控制等领域具有重要的应用前景。传统的电子器件由于缺乏神经元的非线性特性,需要复杂的电路结构和大量的器件才能模拟简单的生物神经元功能,同时功耗也较高。因此,最近研究者们借鉴生物神经元的工作机制,提出了多种基于忆阻器等新型器件的人工脉冲神经元方案。这些方案具有功耗低、结构简单、制备工艺成熟等优点,并且在模拟生物神经元的多种功能等方面取得了显著进展。文章将从人工脉冲神经元的基本原理出发,综述和分析目前已有的各种实现方案。具体来说,将分别介绍基于传统电子器件和基于新型器件的人工脉冲神经元的实现方案,并对其优缺点进行比较。此外,还将介绍不同类型的人工脉冲神经元在实现触觉、视觉、嗅觉、味觉、听觉和温度等神经形态感知方面的应用,并对未来的发展进行展望。希望能够为人工脉冲神经元的研究和应用提供有益的参考和启示。

跨脉冲传播的深度脉冲神经网络训练方法

基于反向传播的脉冲神经网络(SNNs)的训练方法仍面临着诸多问题与挑战,包括脉冲发放过程不可微分、脉冲神经元具有复杂的时空动力过程等。此外,SNNs反向传播训练方法往往没有考虑误差信号在相邻脉冲间的关系,大大降低了网络模型的准确性。为此,提出一种跨脉冲误差传播的深度脉冲神经网络训练方法(cross-spike error backpropagation,CSBP),将神经元的误差反向传播分成脉冲发放时间随突触后膜电位变化关系和相邻脉冲发放时刻点间的依赖关系两种依赖关系。其中,通过前者解决了脉冲不可微分的问题,通过后者明确了脉冲间的依赖关系,使得误差信号能跨脉冲传播,提升了生物合理性。此外,并对早期脉冲残差网络架构存在的模型表示能力不足问题进行研究,通过修改脉冲残余块的结构顺序,进一步提高了网络性能。实验结果表明,所提方法比基于脉冲时间的最优训练算法有着明显的提升,相同架构下,在CIFAR10数据集上提升2.98%,在DVS-CIFAR10数据集上提升2.26%。

机器人类脑智能研究综述

传统机器人经过长时间的研究和发展,已经在生产和生活的多个领域得到了广泛的应用,但在复杂多变的环境中依然缺乏与真实生物类似的灵活性、稳定性和适应能力.类脑智能作为一种新型的机器智能,使用计算建模的方法模拟生物神经系统的各类特性,进而实现对各类信息的推理和决策,近年来受到了学术界的广泛关注.鉴于此,综述了国内外面向机器人系统的类脑智能研究现状,并对类脑智能方法在机器人感知、决策和控制三个研究方向的成果进行了整理、归纳和分析,最后从软硬件层面分别指出了机器人类脑智能目前存在的主要问题和未来的发展方向.

基于自适应时间步脉冲神经网络的高效图像分类

脉冲神经网络(Spiking neural network,SNN)由于具有相对人工神经网络(Artifcial neural network,ANN)更低的计算能耗而受到广泛关注.然而,现有SNN大多基于同步计算模式且往往采用多时间步的方式来模拟动态的信息整合过程,因此带来了推理延迟增大和计算能耗增高等问题,使其在边缘智能设备上的高效运行大打折扣.针对这个问题,本文提出一种自适应时间步脉冲神经网络(Adaptive timestep improved spiking neural network,ATSNN)算法.该算法可以根据不同样本特征自适应选择合适的推理时间步,并通过设计一个时间依赖的新型损失函数来约束不同计算时间步的重要性.与此同时,针对上述ATSNN特点设计一款低能耗脉冲神经网络加速器,支持ATSNN算法在VGG和ResNet等成熟框架上的应用部署.在CIFAR10、CIFAR100、CIFAR10-DVS等标准数据集上软硬件实验结果显示,与当前固定时间步的SNN算法相比,ATSNN算法的精度基本不下降,并且推理延迟减少36.7%~58.7%,计算复杂度减少33.0%~57.0%.在硬件模拟器上的运行结果显示,ATSNN的计算能耗仅为GPU RTX 3090Ti的4.43%~7.88%.显示出脑启发神经形态软硬件的巨大优势.