脉冲神经网络研究现状及展望

Research Advances and Perspectives on Spiking Neural Networks

脉冲神经网络(Spiking Neural Network,SNN)包含具有时序动力学特性的神经元节点、稳态-可塑性平衡的突触结构、功能特异性的网络环路等,高度借鉴了生物启发的局部非监督(如脉冲时序依赖可塑性、短时突触可塑性、局部稳态调节等)、全局弱监督(如多巴胺奖赏学习、基于能量的函数优化等)的生物优化方法,因此具有强大的时空信息表征、异步事件信息处理、网络自组织学习等能力.SNN的研究属于交叉学科,将深入融合脑科学和计算机科学,因此对其研究也可以主要分为两大类:一类是以更好地理解生物系统为最终目的;另一类是以追求卓越计算性能为优化目标.本文首先对当前这两大类SNN的研究进展、研究特点等进行分析,重点介绍基于Spike的多类异步信息编码、基于Motif分布的多亚型复杂网络结构、多层时钟网络自组织计算、神经形态计算芯片的软硬结合等.同时,介绍一种融合生物多尺度、多类型神经可塑性的高效SNN优化策略,使得SNN中的信度分配可以从宏观尺度有效覆盖到微观尺度,如全部的网络输出、网络隐层状态、局部的各个神经节点等,并部分解答生物系统是如何通过局部参数的调优而实现全局网络优化的问题.这将不仅为现有人工智能模型提高其认知能力指明一种可能的生物类优化方向,还为反向促进生命科学中生物神经网络的可塑性研究新发现提供启发.本文认为,脉冲神经网络的发展目标不是构建人工神经网络的生物版本替代品,而是通过突破生物启发的多尺度可塑性优化理论,去粗取精,最终实现具有生物认知计算特色的新一代高效脉冲神经网络模型,使其有望获得更快的学习速度、更小的能量消耗、更强的适应性和更好的可解释性等.

Spiking Neural Network(SNN)contains neurons with sequential dynamics,synapses with plastic stability,and circuits with specific cognitive functions.SNN is biologically-plausible and can be tuned by integrating local-scale unsupervised learning rules(e.g.,Spike Timing-Dependent Plasticity,Short-Term Plasticity,local equilibrium adjustment of membrane potential)and global-scale weak-supervised learning rules(e.g.,dopamine-based reward learning,energy-based learning).Hence,it is powerful on spatially-temporal information representation,asynchronous processing of event-based information,and self-organized learning with dynamic topologies.SNN belongs to cross-discipline research areas of brain science and computer science.Hence,the research on it can be divided into two main types.One type is designed to understand better the biological system,where detailed biologically-realistic neural models are used without further consideration of computational efficiency.The other type is constructed to pursue superior computational performance,where only limited features of SNN are retained,and some efficient but not biologically-plausible tuning methods are still used,such as different versions of backpropagation.A detailed analysis of the research advances and model characteristics of these two types of efforts is given,including the following aspects:Firstly,the multi-type information encoding at neuronal scales is given,with event-based signal processing characteristics;Secondly,the multi-scale sparseness of network structures is defined with different subtypes of network motifs;Thirdly,the self-organized computation is shown at multi-scale clocks,from micro-scale at neurons(or synapses)to macro-scale at circuits;Fourthly,some vital functional characteristics of SNN are introduced,including energy-efficient computation(with spikes and learning rules)and robust computation(e.g.,anti-environmental noises);Fifthly,the integration of SNN with neuromorphic hardware is shown for efficient non-Von-Neumann computation.After that,we will introduce a biologically-plausible strategy for well-tuning SNN by integrating multi-scale and multi-type plasticity rules inspired by natural neural networks and fine-tuning processes of state-of-the-art ANNs.This strategy provides an alternative effort for the efficient credit assignment for SNN,covering the whole network neurons,from readout,locally-hidden,to input neurons.It also gives us hints on answering the critical question,i.e.,how biological neural networks can handle global network-tuning problems by integrating different types of local plasticity rules.These integrative principles will give SNN a possibly right tuning direction towards efficiently cognitive computation.Simultaneously,the success of SNN will also give inspirations back to the findings of new plasticity rules in natural neural networks.We think the goal of SNN is not just working as a biological candidate of ANNs,but constructing a new generation of effective artificial-intelligence models with characteristics of biologically-plausible cognitive computation by integrating theoretical breakthroughs in biology-inspired multi-scale plasticity principles,towards the faster learning convergence,lower energy cost,stronger adaptability,higher robust computation,and also better interpretability.