Memory Analysis for Memristors and Memristive Recurrent Neural Networks

Traditional recurrent neural networks are composed of capacitors, inductors, resistors, and operational amplifiers.Memristive neural networks are constructed by replacing resistors with memristors. This paper focuses on the memory analysis,i.e. the initial value computation, of memristors. Firstly, we present the memory analysis for a single memristor based on memristors’ mathematical models with linear and nonlinear drift.Secondly, we present the memory analysis for two memristors in series and parallel. Thirdly, we point out the difference between traditional neural networks and those that are memristive. Based on the current and voltage relationship of memristors, we use mathematical analysis and SPICE simulations to demonstrate the validity of our methods.

Lightweight and highly robust memristor-based hybrid neural networks for electroencephalogram signal processing

Memristor-based neuromorphic computing shows great potential for high-speed and high-throughput signal processing applications,such as electroencephalogram(EEG)signal processing.Nonetheless,the size of one-transistor one-resistor(1T1R)memristor arrays is limited by the non-ideality of the devices,which prevents the hardware implementation of large and complex networks.In this work,we propose the depthwise separable convolution and bidirectional gate recurrent unit(DSC-BiGRU)network,a lightweight and highly robust hybrid neural network based on 1T1R arrays that enables efficient processing of EEG signals in the temporal,frequency and spatial domains by hybridizing DSC and BiGRU blocks.The network size is reduced and the network robustness is improved while ensuring the network classification accuracy.In the simulation,the measured non-idealities of the 1T1R array are brought into the network through statistical analysis.Compared with traditional convolutional networks,the network parameters are reduced by 95%and the network classification accuracy is improved by 21%at a 95%array yield rate and 5%tolerable error.This work demonstrates that lightweight and highly robust networks based on memristor arrays hold great promise for applications that rely on low consumption and high efficiency.

Design of multilayer cellular neural network based on memristor crossbar and its application to edge detection

Memristor with memory properties can be applied to connection points(synapses)between cells in a cellular neural network(CNN).This paper highlights memristor crossbar-based multilayer CNN(MCM-CNN)and its application to edge detection.An MCM-CNN is designed by adopting a memristor crossbar composed of a pair of memristors.MCM-CNN based on the memristor crossbar with changeable weight is suitable for edge detection of a binary image and a color image considering its characteristics of programmablization and compactation.Figure of merit(FOM)is introduced to evaluate the proposed structure and several traditional edge detection operators for edge detection results.Experiment results show that the FOM of MCM-CNN is three times more than that of the traditional edge detection operators.

Advances in memristor based artificial neuron fabrication-materials,models,and applications

Spiking neural network(SNN),widely known as the third-generation neural network,has been frequently investigated due to its excellent spatiotemporal information processing capability,high biological plausibility,and low energy consumption characteristics.Analogous to the working mechanism of human brain,the SNN system transmits information through the spiking action of neurons.Therefore,artificial neurons are critical building blocks for constructing SNN in hardware.Memristors are drawing growing attention due to low consumption,high speed,and nonlinearity characteristics,which are recently introduced to mimic the functions of biological neurons.Researchers have proposed multifarious memristive materials including organic materials,inorganic materials,or even two-dimensional materials.Taking advantage of the unique electrical behavior of these materials,several neuron models are successfully implemented,such as Hodgkin–Huxley model,leaky integrate-and-fire model and integrate-and-fire model.In this review,the recent reports of artificial neurons based on memristive devices are discussed.In addition,we highlight the models and applications through combining artificial neuronal devices with sensors or other electronic devices.Finally,the future challenges and outlooks of memristor-based artificial neurons are discussed,and the development of hardware implementation of brain-like intelligence system based on SNN is also prospected.

One memristor–one electrolyte-gated transistor-based high energy-efficient dropout neuronal units

Artificial neural networks(ANN) have been extensively researched due to their significant energy-saving benefits.Hardware implementations of ANN with dropout function would be able to avoid the overfitting problem. This letter reports a dropout neuronal unit(1R1T-DNU) based on one memristor–one electrolyte-gated transistor with an ultralow energy consumption of 25 p J/spike. A dropout neural network is constructed based on such a device and has been verified by MNIST dataset, demonstrating high recognition accuracies(> 90%) within a large range of dropout probabilities up to40%. The running time can be reduced by increasing dropout probability without a significant loss in accuracy. Our results indicate the great potential of introducing such 1R1T-DNUs in full-hardware neural networks to enhance energy efficiency and to solve the overfitting problem.

Exponential synchronization of coupled memristive neural networks via pinning control

This paper is concerned with the exponential synchronization problem of coupled memristive neural networks. In contrast to general neural networks, memristive neural networks exhibit state-dependent switching behaviors due to the physical properties of memristors. Under a mild topology condition, it is proved that a small fraction of controlled sub- systems can efficiently synchronize the coupled systems. The pinned subsystems are identified via a search algorithm. Moreover, the information exchange network needs not to be undirected or strongly connected. Finally, two numerical simulations are performed to verify the usefulness and effectiveness of our results.

Finite-time Mittag-Leffler synchronization of fractional-order complex-valued memristive neural networks with time delay

Without dividing the complex-valued systems into two real-valued ones, a class of fractional-order complex-valued memristive neural networks(FCVMNNs) with time delay is investigated. Firstly, based on the complex-valued sign function, a novel complex-valued feedback controller is devised to research such systems. Under the framework of Filippov solution, differential inclusion theory and Lyapunov stability theorem, the finite-time Mittag-Leffler synchronization(FTMLS) of FCVMNNs with time delay can be realized. Meanwhile, the upper bound of the synchronization settling time(SST) is less conservative than previous results. In addition, by adjusting controller parameters, the global asymptotic synchronization of FCVMNNs with time delay can also be realized, which improves and enrich some existing results. Lastly,some simulation examples are designed to verify the validity of conclusions.

Synchronization of Stochastic Memristive Neural Networks with Retarded and Advanced Argument

In this paper, we discuss the driving-response synchronization problem for two memristive neural networks with retarded and advanced arguments under the condition of additional noise. The control law is related to the linear time-delay feedback term, and the discontinuous feedback term. Moreover, the random different equation is used to prove the stability of this theory. At the end, the simulation results verify the correctness of the theoretical results.

Finite-time Mittag-Leffler synchronization of fractional-order delayed memristive neural networks with parameters uncertainty and discontinuous activation functions

The finite-time Mittag-Leffler synchronization is investigated for fractional-order delayed memristive neural networks(FDMNN)with parameters uncertainty and discontinuous activation functions.The relevant results are obtained under the framework of Filippov for such systems.Firstly,the novel feedback controller,which includes the discontinuous functions and time delays,is proposed to investigate such systems.Secondly,the conditions on finite-time Mittag-Leffler synchronization of FDMNN are established according to the properties of fractional-order calculus and inequality analysis technique.At the same time,the upper bound of the settling time for Mittag-Leffler synchronization is accurately estimated.In addition,by selecting the appropriate parameters of the designed controller and utilizing the comparison theorem for fractional-order systems,the global asymptotic synchronization is achieved as a corollary.Finally,a numerical example is given to indicate the correctness of the obtained conclusions.

Mixed H_(∞)/Passive Exponential Synchronization for Delayed Memristive Neural Networks with Switching Event-Triggered Control

This paper is devoted to event-triggered synchronization of delayed memristive neural networks with H∞and passivity performance.The aim is to guarantee the exponential synchronization and mixed H∞and passivity control for memristive neural networks by using event-triggered control.Firstly,a switching system is constructed under the event-triggered control strategy.Then,by adopting a piece-wise Lyapunov functional,a sufficient condition is established for the exponential synchronization and mixed H_(∞)and passivity performance.Moreover,an event-triggered controller design scheme is proposed using matrix decoupling method.Finally,the effectiveness of the designed controller is exemplified by a numerical example.

Memristor-based spiking neural networks:cooperative development of neural network architecture/algorithms and memristors

Inspired by the structure and principles of the human brain,spike neural networks(SNNs)appear as the latest generation of artificial neural networks,attracting significant and universal attention due to their remarkable low-energy transmission by pulse and powerful capability for large-scale parallel computation.Current research on artificial neural networks gradually change from software simulation into hardware implementation.However,such a process is fraught with challenges.In particular,memristors are highly anticipated hardware candidates owing to their fastprogramming speed,low power consumption,and compatibility with the complementary metal–oxide semiconductor(CMOS)technology.In this review,we start from the basic principles of SNNs,and then introduced memristor-based technologies for hardware implementation of SNNs,and further discuss the feasibility of integrating customized algorithm optimization to promote efficient and energy-saving SNN hardware systems.Finally,based on the existing memristor technology,we summarize the current problems and challenges in this field.

Memristor-based vector neural network architecture

Vector neural network(VNN)is one of the most important methods to process interval data.However,the VNN,which contains a great number of multiply-accumulate(MAC)operations,often adopts pure numerical calculation method,and thus is difficult to be miniaturized for the embedded applications.In this paper,we propose a memristor based vector-type backpropagation(MVTBP)architecture which utilizes memristive arrays to accelerate the MAC operations of interval data.Owing to the unique brain-like synaptic characteristics of memristive devices,e.g.,small size,low power consumption,and high integration density,the proposed architecture can be implemented with low area and power consumption cost and easily applied to embedded systems.The simulation results indicate that the proposed architecture has better identification performance and noise tolerance.When the device precision is 6 bits and the error deviation level(EDL)is 20%,the proposed architecture can achieve an identification rate,which is about 92%higher than that for interval-value testing sample and 81%higher than that for scalar-value testing sample.

Complex dynamic behaviors in hyperbolic-type memristor-based cellular neural network

This paper presents a new hyperbolic-type memristor model,whose frequency-dependent pinched hysteresis loops and equivalent circuit are tested by numerical simulations and analog integrated operational amplifier circuits.Based on the hyperbolic-type memristor model,we design a cellular neural network(CNN)with 3-neurons,whose characteristics are analyzed by bifurcations,basins of attraction,complexity analysis,and circuit simulations.We find that the memristive CNN can exhibit some complex dynamic behaviors,including multi-equilibrium points,state-dependent bifurcations,various coexisting chaotic and periodic attractors,and offset of the positions of attractors.By calculating the complexity of the memristor-based CNN system through the spectral entropy(SE)analysis,it can be seen that the complexity curve is consistent with the Lyapunov exponent spectrum,i.e.,when the system is in the chaotic state,its SE complexity is higher,while when the system is in the periodic state,its SE complexity is lower.Finally,the realizability and chaotic characteristics of the memristive CNN system are verified by an analog circuit simulation experiment.

Memristor-based multi-synaptic spiking neuron circuit for spiking neural network

Spiking neural networks(SNNs) are widely used in many fields because they work closer to biological neurons.However,due to its computational complexity,many SNNs implementations are limited to computer programs.First,this paper proposes a multi-synaptic circuit(MSC) based on memristor,which realizes the multi-synapse connection between neurons and the multi-delay transmission of pulse signals.The synapse circuit participates in the calculation of the network while transmitting the pulse signal,and completes the complex calculations on the software with hardware.Secondly,a new spiking neuron circuit based on the leaky integrate-and-fire(LIF) model is designed in this paper.The amplitude and width of the pulse emitted by the spiking neuron circuit can be adjusted as required.The combination of spiking neuron circuit and MSC forms the multi-synaptic spiking neuron(MSSN).The MSSN was simulated in PSPICE and the expected result was obtained,which verified the feasibility of the circuit.Finally,a small SNN was designed based on the mathematical model of MSSN.After the SNN is trained and optimized,it obtains a good accuracy in the classification of the IRIS-dataset,which verifies the practicability of the design in the network.

Dynamics analysis and cryptographic implementation of a fractional-order memristive cellular neural network model

Due to the fact that a memristor with memory properties is an ideal electronic component for implementation of the artificial neural synaptic function,a brand-new tristable locally active memristor model is first proposed in this paper.Here,a novel four-dimensional fractional-order memristive cellular neural network(FO-MCNN)model with hidden attractors is constructed to enhance the engineering feasibility of the original CNN model and its performance.Then,its hardware circuit implementation and complicated dynamic properties are investigated on multi-simulation platforms.Subsequently,it is used toward secure communication application scenarios.Taking it as the pseudo-random number generator(PRNG),a new privacy image security scheme is designed based on the adaptive sampling rate compressive sensing(ASR-CS)model.Eventually,the simulation analysis and comparative experiments manifest that the proposed data encryption scheme possesses strong immunity against various security attack models and satisfactory compression performance.

Coexistence behavior of asymmetric attractors in hyperbolic-type memristive Hopfield neural network and its application in image encryption

The neuron model has been widely employed in neural-morphic computing systems and chaotic circuits.This study aims to develop a novel circuit simulation of a three-neuron Hopfield neural network(HNN)with coupled hyperbolic memristors through the modification of a single coupling connection weight.The bistable mode of the hyperbolic memristive HNN(mHNN),characterized by the coexistence of asymmetric chaos and periodic attractors,is effectively demonstrated through the utilization of conventional nonlinear analysis techniques.These techniques include bifurcation diagrams,two-parameter maximum Lyapunov exponent plots,local attractor basins,and phase trajectory diagrams.Moreover,an encryption technique for color images is devised by leveraging the mHNN model and asymmetric structural attractors.This method demonstrates significant benefits in correlation,information entropy,and resistance to differential attacks,providing strong evidence for its effectiveness in encryption.Additionally,an improved modular circuit design method is employed to create the analog equivalent circuit of the memristive HNN.The correctness of the circuit design is confirmed through Multisim simulations,which align with numerical simulations conducted in Matlab.

Hybrid memristor-CMOS neurons for in-situ learning in fully hardware memristive spiking neural networks

Spiking neural network,inspired by the human brain,consisting of spiking neurons and plastic synapses,is a promising solution for highly efficient data processing in neuromorphic computing.Recently,memristor-based neurons and synapses are becoming intriguing candidates to build spiking neural networks in hardware,owing to the close resemblance between their device dynamics and the biological counterparts.However,the functionalities of memristor-based neurons are currently very limited,and a hardware demonstration of fully memristor-based spiking neural networks supporting in-situ learning is very challenging.Here,a hybrid spiking neuron combining a memristor with simple digital circuits is designed and implemented in hardware to enhance neuron functions.The hybrid neuron with memristive dynamics not only realizes the basic leaky integrate-and-fire neuron function but also enables the in-situ tuning of the connected synaptic weights.Finally,a fully hardware spiking neural network with the hybrid neurons and memristive synapses is experimentally demonstrated for the first time,and in-situ Hebbian learning is achieved with this network.This work opens up a way towards the implementation of spiking neurons,supporting in-situ learning for future neuromorphic computing systems.

Multiply accumulate operations in memristor crossbar arrays for analog computing

Memristors are now becoming a prominent candidate to serve as the building blocks of non-von Neumann inmemory computing architectures.By mapping analog numerical matrices into memristor crossbar arrays,efficient multiply accumulate operations can be performed in a massively parallel fashion using the physics mechanisms of Ohm’s law and Kirchhoff’s law.In this brief review,we present the recent progress in two niche applications:neural network accelerators and numerical computing units,mainly focusing on the advances in hardware demonstrations.The former one is regarded as soft computing since it can tolerant some degree of the device and array imperfections.The acceleration of multiple layer perceptrons,convolutional neural networks,generative adversarial networks,and long short-term memory neural networks are described.The latter one is hard computing because the solving of numerical problems requires high-precision devices.Several breakthroughs in memristive equation solvers with improved computation accuracies are highlighted.Besides,other nonvolatile devices with the capability of analog computing are also briefly introduced.Finally,we conclude the review with discussions on the challenges and opportunities for future research toward realizing memristive analog computing machines.

Optically modulated dual-mode memristor arrays based on core-shell CsPbBr_(3)@graphdiyne nanocrystals for fully memristive neuromorphic computing hardware

Artificial synapses and neurons are crucial milestones for neuromorphic computing hardware,and memristors with resistive and threshold switching characteristics are regarded as the most promising candidates for the construction of hardware neural networks.However,most of the memristors can only operate in one mode,that is,resistive switching or threshold switching,and distinct memristors are required to construct fully memristive neuromorphic computing hardware,making it more complex for the fabrication and integration of the hardware.Herein,we propose a flexible dual-mode memristor array based on core–shell CsPbBr3@graphdiyne nanocrystals,which features a 100%transition yield,small cycle-to-cycle and device-to-device variability,excellent flexibility,and environmental stability.Based on this dual-mode memristor,homo-material-based fully memristive neuromorphic computing hardware—a power-free artificial nociceptive signal processing system and a spiking neural network—are constructed for the first time.Our dual-mode memristors greatly simplify the fabrication and integration of fully memristive neuromorphic systems.