Electrically programmable phase-change photonic memory for optical neural networks with nanoseconds in situ training capability

Optical neural networks (ONNs), enabling low latency and high parallel data processing withoutelectromagnetic interference, have become a viable player for fast and energy-efficient processing andcalculation to meet the increasing demand for hash rate. Photonic memories employing nonvolatile phase-change materials could achieve zero static power consumption, low thermal cross talk, large-scale, andhigh-energy-efficient photonic neural networks. Nevertheless, the switching speed and dynamic energyconsumption of phase-change material-based photonic memories make them inapplicable for in situ training.Here, by integrating a patch of phase change thin film with a PIN-diode-embedded microring resonator,a bifunctional photonic memory enabling both 5-bit storage and nanoseconds volatile modulation wasdemonstrated. For the first time, a concept is presented for electrically programmable phase-changematerial-driven photonic memory integrated with nanosecond modulation to allow fast in situ training and zerostatic power consumption data processing in ONNs. ONNs with an optical convolution kernel constructedby our photonic memory theoretically achieved an accuracy of predictions higher than 95% when testedby the MNIST handwritten digit database. This provides a feasible solution to constructing large-scalenonvolatile ONNs with high-speed in situ training capability.

STABILITY OF BIDIRECTIONAL ASSOCIATIVE MEMORY NEURAL NETWORKS WITH DELAYS

In this paper the globally asymptotic stability of more general two-layer nonlinear feedback associative memory neural networks with time delays is examined. The sufficient conditions of existence, uniqueness and globally asymptotic stability of the equilibrum position are given. Finally, two interesting examples to illustrate the theory are given.

QUALITATIVE ANALYSIS OF BIDIRECTIONAL ASSOCIATIVE MEMORY NEURAL NETWORKS

In this paper, the global exponential stability of an equilibrium position for general bidirectional associative memory neural networks are studied. The sufficient conditions of existence and uniqueness of the equilibrium position are given. The method of energy function is examined. Two examples are given to illustrate the theory.

Multi-Valued Associative Memory Neural Network

A novel learning method for multi-valued associative memory network is introduced, which is based on Hebb rule, but utilizes more information. According to the current probe vector, the connection weights matrix could be chosen dynamically. Double-valued and multi-valued associative memory are all realized in our simulation experiment. The experimental results show that the method could enhance the associative success rate.

CHAOTIC NEURAL NETWORK FOR ASSOCIATIVE MEMORY

Based on current research on applications of chaotic neuron network for information processing, the stability and convergence of chaotic neuron network are proved from the viewpoint of energy function. Moreover, a new auto-associative matrix is devised for artificial neural network composed of chaotic neurons, thus, an improved chaotic neuron network for associative memory is built up. Finally, the associative recalling process of the network is analyzed in detail and explanations of improvement are given.

The FAM(Fuzzy Asociative Memory)neural network model and its application in earthquake prediction

FAM(Fuzzy Associative Memory) Network Model, FAM Adaptive Learning Algorithm and Principal of FAM Inference Machine are introduced, and successfully application to ″New Generation Expert System for Earthquake Prediction″ (NGESEP). This system has good function for knowledge learning without disadvantages of neural network, which the learned knowledge implied in network is difficult to be understood or interpreted by expert system.

A Hopfield-like hippocampal CA3 neural network model for studying associative memory in Alzheimer's disease

Associative memory, one of the major cognitive functions in the hippocampal CA3 region, includes auto-associative memory and hetero-associative memory. Many previous studies have shown that Alzheimer＇s disease （AD） can lead to loss of functional synapses in the central nervous system, and associative memory functions in patients with AD are often impaired, but few studies have addressed the effect of AD on hetero-associative memory in the hippocampal CA3 region. In this study, based on a simplified anatomical structure and synaptic connections in the hippocampal CA3 region, a three-layered Hopfield-like neural network model of hippocampal CA3 was proposed and then used to simulate associative memory functions in three circumstances： normal, synaptic deletion and synaptic compensation, according to Ruppin＇s synaptic deletion and compensation theory. The influences of AD on hetero-associative memory were further analyzed. The simulated results showed that the established three-layered Hopfield-like neural network model of hippocampal CA3 has both auto-associative and hetero-associative memory functions. With increasing synaptic deletion level, both associative memory functions were gradually impaired and the mean firing rates of the neurons within the network model were decreased. With gradual increasing synaptic compensation, the associative memory functions of the network were improved and the mean firing rates were increased. The simulated results suggest that the Hopfield-like neural network model can effectively simulate both associative memory functions of the hippocampal CA3 region. Synaptic deletion affects both auto-associative and hetero-associative memory functions in the hippocampal CA3 region, and can also result in memory dysfunction. To some extent, synaptic compensation measures can offset two kinds of associative memory dysfunction caused by synaptic deletion in the hippocampal CA3 area.

GLOBAL DYNAMICS OF DELAYED BIDIRECTIONAL ASSOCIATIVE MEMORY (BAM) NEURAL NETWORKS

Without assuming the smoothness,monotonicity and boundedness of the activation functions, some novel criteria on the existence and global exponential stability of equilibrium point for delayed bidirectional associative memory (BAM) neural networks are established by applying the Liapunov functional methods and matrix_algebraic techniques. It is shown that the new conditions presented in terms of a nonsingular M matrix described by the networks parameters,the connection matrix and the Lipschitz constant of the activation functions,are not only simple and practical,but also easier to check and less conservative than those imposed by similar results in recent literature.

Double-pattern associative memory neural network with pattern loop

A double-pattern associative memory neural network with “pattern loop” is proposed. It can store 2N bit bipolar binary patterns up to the order of 2 2N , retrieve part or all of the stored patterns which all have the minimum Hamming distance with input pattern, completely eliminate spurious patterns, and has higher storing efficiency and reliability than conventional associative memory. The length of a pattern stored in this associative memory can be easily extended from 2N to kN.

Memory Chain in a Chaotic Autoassociative Neural Network

Memory chain is observed in a chaotic autoassociative neural network. The network recalls first stored pattern from a fragment of a memory, stays at this pattern for a while, transits to the second stored pattern that overlaps with the first recalled pattern.Then it stays at the second recalled pattern for a while, transits to the third stored pattern that overlaps with the second recalled pattern, and so on. Thus a memory chain is generated. The memory chain ends with the pattern that overlaps no other stored patten. This phenomenon is similar to the way of recalling process of human beings in some respects.

An Incremental Time-delay Neural Network for Dynamical Recurrent Associative Memory

An incremental time-delay neural network based on synapse growth, which is suitable for dynamic control and learning of autonomous robots, is proposed to improve the learning and retrieving performance of dynamical recurrent associative memory architecture. The model allows steady and continuous establishment of associative memory for spatio-temporal regularities and time series in discrete sequence of inputs. The inserted hidden units can be taken as the long-term memories that expand the capacity of network and sometimes may fade away under certain condition. Preliminary experiment has shown that this incremental network may be a promising approach to endow autonomous robots with the ability of adapting to new data without destroying the learned patterns. The system also benefits from its potential chaos character for emergence.

An oscillatory neural network and dynamical associative memory

Ａｒｔｉｆｉｃｉａｌｎｅｕｒａｌｎｅｔｗｏｒｋｈａｓｒｅｃｅｎｔｌｙｒｅｃｅｉｖｅｄｃｏｎｓｉｄｅｒａｂｌｅａｔｅｎｔｉｏｎｆｒｏｍｖａｒｉｏｕｓｒｅｓｅａｒｃｈｆｉｅｌｄｓ．Ｍｕｃｈｅｆｏｒｔｈａｓｂｅｅｎｍａｄｅｔｏｗａｒｄｓｔｈｅｕｎｄｅｒｓｔ...

Application of Neural Network in Fault Location of Optical Transport Network

Due to the increasing variety of information and services carried by optical networks, the survivability of network becomes an important problem in current research. The fault location of OTN is of great significance for studying the survivability of optical networks. Firstly, a three-channel network model is established and analyzing common alarm data, the fault monitoring points and common fault points are carried out. The artificial neural network is introduced into the fault location field of OTN and it is used to judge whether the possible fault point exists or not. But one of the obvious limitations of general neural networks is that they receive a fixedsize vector as input and produce a fixed-size vector as the output. Not only that, these models is even fixed for mapping operations (for example, the number of layers in the model). The difference between the recurrent neural network and general neural networks is that it can operate on the sequence. In spite of the fact that the gradient disappears and the gradient explodes still exist in the neural network, the method of gradient shearing or weight regularization is adopted to solve this problem, and choose the LSTM (long-short term memory networks) to locate the fault. The output uses the concept of membership degree of fuzzy theory to express the possible fault point with the probability from 0 to 1. Priority is given to the treatment of fault points with high probability. The concept of F-Measure is also introduced, and the positioning effect is measured by using location time, MSE and F-Measure. The experiment shows that both LSTM and BP neural network can locate the fault of optical transport network well, but the overall effect of LSTM is better. The localization time of LSTM is shorter than that of BP neural network, and the F1-score of LSTM can reach 0.961566888396156 after 45 iterations, which meets the accuracy and real-time requirements of fault location. Therefore, it has good application prospect and practical value to introduce neural network into the fault location field of optical transport network.

Synthesization of high-capacity auto-associative memories using complex-valued neural networks

In this paper, a novel design procedure is proposed for synthesizing high-capacity auto-associative memories based on complex-valued neural networks with real-imaginary-type activation functions and constant delays. Stability criteria dependent on external inputs of neural networks are derived. The designed networks can retrieve the stored patterns by external inputs rather than initial conditions. The derivation can memorize the desired patterns with lower-dimensional neural networks than real-valued neural networks, and eliminate spurious equilibria of complex-valued neural networks. One numerical example is provided to show the effectiveness and superiority of the presented results.

Almost periodic solutions of memristive multidirectional associative memory neural networks with mixed time delays

Traditional biological neural networks cannot simulate the real situation of the abrupt synaptic connections between neurons while modeling associative memory of human brains.In this paper,the memristive multidirectional associative memory neural networks(MAMNNs)with mixed time-varying delays are investigated in the sense of Filippov solution.First,three steps are given to prove the existence of the almost periodic solution.Two new lemmas are proposed to prove the boundness of the solution and the asymptotical almost periodicity of the solution by constructing Lyapunov function.Second,the uniqueness and global exponential stability of the almost periodic solution of memristive MAMNNs are investigated by a new Lyapunov function.The sufficient conditions guaranteeing the properties of almost periodic solution are derived based on the relevant definitions,Halanay inequality and Lyapunov function.The investigation is an extension of the research on the periodic solution and almost periodic solution of bidirectional associative memory neural networks.Finally,numerical examples with simulations are presented to show the validity of the main results.

A Novel Multivalued Associative Memory System

Globally coupled map (GCM) model can evolve through chaotic searching into several stable periodic orbits under properly controlled parameters. This can be exploited in information processing such as associative memory and optimization. In this paper, we propose a novel covariance learning rule for multivalue patterns and apply it in memorization of gray scale images based on modified GCM model (S GCM). Analysis of retrieval results are given finally.

BIDIRECTIONAL ASSOCIATIVE MEMORY ENSEMBLE

The multiple classifier system （MCS）, composed of multiple diverse classifiers or feed-forward neural networks, can significantly improve the classification or generalization ability of a single classifier. Enlightened by the fundamental idea of MCS, the ensemble is introduced into the quick learning for bidirectional associative memory （QLBAM） to construct a BAM ensemble, for improving the storage capacity and the error-correction capability without destroying the simple structure of the component BAM. Simulations show that, with an appropriate ＂overproduce and choose＂ strategy or ＂thinning＂ algorithm, the proposed BAM ensemble significantly outperforms the single QLBAM in both storage capacity and noise-tolerance capability.

LMI-based approach for global asymptotic stability analysis of continuous BAM neural networks

Studies on the stability of the equilibrium points of continuous bidirectional associative memory (BAM) neural network have yielded many useful results. A novel neural network model called standard neural network model (SNNM) is ad- vanced. By using state affine transformation, the BAM neural networks were converted to SNNMs. Some sufficient conditions for the global asymptotic stability of continuous BAM neural networks were derived from studies on the SNNMs’ stability. These conditions were formulated as easily verifiable linear matrix inequalities (LMIs), whose conservativeness is relatively low. The approach proposed extends the known stability results, and can also be applied to other forms of recurrent neural networks (RNNs).

ESTIMATION OF ATTRACTION DOMAIN AND EXPONENTIAL CONVERGENCE RATE OF CONTINUOUS FEEDBACK ASSOCIATIVE MEMORY AND ITS APPLICATIONS

The attraction domains of memory patterns and exponential convergence rate of the network trajectories to memory patterns for continuous feedback associative memory are estimated. These results can be used for evaluation of error-correction capability and the synthesis procedures for continuous-time associative memory neural networks.