A Kind of Second-Order Learning Algorithm Based on Generalized Cost Criteria in Multi-Layer Feed-Forward Neural Networks

A kind of second order algorithm--recursive approximate Newton algorithm was given by Karayiannis. The algorithm was simplified when it was formulated. Especially, the simplification to matrix Hessian was very reluctant, which led to the loss of valuable information and affected performance of the algorithm to certain extent. For multi layer feed forward neural networks, the second order back propagation recursive algorithm based generalized cost criteria was proposed. It is proved that it is equivalent to Newton recursive algorithm and has a second order convergent rate. The performance and application prospect are analyzed. Lots of simulation experiments indicate that the calculation of the new algorithm is almost equivalent to the recursive least square multiple algorithm. The algorithm and selection of networks parameters are significant and the performance is more excellent than BP algorithm and the second order learning algorithm that was given by Karayiannis.

Some Features of Neural Networks as Nonlinearly Parameterized Models of Unknown Systems Using an Online Learning Algorithm

This paper deals with deriving the properties of updated neural network model that is exploited to identify an unknown nonlinear system via the standard gradient learning algorithm. The convergence of this algorithm for online training the three-layer neural networks in stochastic environment is studied. A special case where an unknown nonlinearity can exactly be approximated by some neural network with a nonlinear activation function for its output layer is considered. To analyze the asymptotic behavior of the learning processes, the so-called Lyapunov-like approach is utilized. As the Lyapunov function, the expected value of the square of approximation error depending on network parameters is chosen. Within this approach, sufficient conditions guaranteeing the convergence of learning algorithm with probability 1 are derived. Simulation results are presented to support the theoretical analysis.

Volterra Feedforward Neural Networks:Theory and Algorithms

The Volterra feedforward neural network with nonlinear interconnections and related homotopy learning algorithm are proposed in the paper. It is shown that Volterra neural network and the homolopy learning algorithms are significant potentials in nonlinear approximation ability,convergent speeds and global optimization than the classical neural networks and the standard BP algorithm, and related computer simulations and theoretical analysis are given too.

A Review of Computing with Spiking Neural Networks

Artificial neural networks(ANNs)have led to landmark changes in many fields,but they still differ significantly fromthemechanisms of real biological neural networks and face problems such as high computing costs,excessive computing power,and so on.Spiking neural networks(SNNs)provide a new approach combined with brain-like science to improve the computational energy efficiency,computational architecture,and biological credibility of current deep learning applications.In the early stage of development,its poor performance hindered the application of SNNs in real-world scenarios.In recent years,SNNs have made great progress in computational performance and practicability compared with the earlier research results,and are continuously producing significant results.Although there are already many pieces of literature on SNNs,there is still a lack of comprehensive review on SNNs from the perspective of improving performance and practicality as well as incorporating the latest research results.Starting from this issue,this paper elaborates on SNNs along the complete usage process of SNNs including network construction,data processing,model training,development,and deployment,aiming to provide more comprehensive and practical guidance to promote the development of SNNs.Therefore,the connotation and development status of SNNcomputing is reviewed systematically and comprehensively from four aspects:composition structure,data set,learning algorithm,software/hardware development platform.Then the development characteristics of SNNs in intelligent computing are summarized,the current challenges of SNNs are discussed and the future development directions are also prospected.Our research shows that in the fields of machine learning and intelligent computing,SNNs have comparable network scale and performance to ANNs and the ability to challenge large datasets and a variety of tasks.The advantages of SNNs over ANNs in terms of energy efficiency and spatial-temporal data processing have been more fully exploited.And the development of programming and deployment tools has lowered the threshold for the use of SNNs.SNNs show a broad development prospect for brain-like computing.

A Modified Algorithm for Feedforward Neural Networks

As a most popular learning algorithm for the feedforward neural networks, the classic BP algorithm has its many shortages. To overcome some of the shortages, a modified learning algorithm is proposed in the article. And the simulation result illustrate the modified algorithm is more effective and practicable.

Application of Random Search Methods in the Determination of Learning Rate for Training Container Dwell Time Data Using Artificial Neural Networks

Purpose: This study aimed to enhance the prediction of container dwell time, a crucial factor for optimizing port operations, resource allocation, and supply chain efficiency. Determining an optimal learning rate for training Artificial Neural Networks (ANNs) has remained a challenging task due to the diverse sizes, complexity, and types of data involved. Design/Method/Approach: This research used a RandomizedSearchCV algorithm, a random search approach, to bridge this knowledge gap. The algorithm was applied to container dwell time data from the TOS system of the Port of Tema, which included 307,594 container records from 2014 to 2022. Findings: The RandomizedSearchCV method outperformed standard training methods both in terms of reducing training time and improving prediction accuracy, highlighting the significant role of the constant learning rate as a hyperparameter. Research Limitations and Implications: Although the study provides promising outcomes, the results are limited to the data extracted from the Port of Tema and may differ in other contexts. Further research is needed to generalize these findings across various port systems. Originality/Value: This research underscores the potential of RandomizedSearchCV as a valuable tool for optimizing ANN training in container dwell time prediction. It also accentuates the significance of automated learning rate selection, offering novel insights into the optimization of container dwell time prediction, with implications for improving port efficiency and supply chain operations.

Model algorithm control using neural networks for input delayed nonlinear control system

The performance of the model algorithm control method is partially based on the accuracy of the system＇s model. It is difficult to obtain a good model of a nonlinear system, especially when the nonlinearity is high. Neural networks have the ability to ＂learn＂the characteristics of a system through nonlinear mapping to represent nonlinear functions as well as their inverse functions. This paper presents a model algorithm control method using neural networks for nonlinear time delay systems. Two neural networks are used in the control scheme. One neural network is trained as the model of the nonlinear time delay system, and the other one produces the control inputs. The neural networks are combined with the model algorithm control method to control the nonlinear time delay systems. Three examples are used to illustrate the proposed control method. The simulation results show that the proposed control method has a good control performance for nonlinear time delay systems.

Soft measurement model of ring's dimensions for vertical hot ring rolling process using neural networks optimized by genetic algorithm

Vertical hot ring rolling(VHRR) process has the characteristics of nonlinearity,time-variation and being susceptible to disturbance.Furthermore,the ring's growth is quite fast within a short time,and the rolled ring's position is asymmetrical.All of these cause that the ring's dimensions cannot be measured directly.Through analyzing the relationships among the dimensions of ring blanks,the positions of rolls and the ring's inner and outer diameter,the soft measurement model of ring's dimensions is established based on the radial basis function neural network(RBFNN).A mass of data samples are obtained from VHRR finite element(FE) simulations to train and test the soft measurement NN model,and the model's structure parameters are deduced and optimized by genetic algorithm(GA).Finally,the soft measurement system of ring's dimensions is established and validated by the VHRR experiments.The ring's dimensions were measured artificially and calculated by the soft measurement NN model.The results show that the calculation values of GA-RBFNN model are close to the artificial measurement data.In addition,the calculation accuracy of GA-RBFNN model is higher than that of RBFNN model.The research results suggest that the soft measurement NN model has high precision and flexibility.The research can provide practical methods and theoretical guidance for the accurate measurement of VHRR process.

Combining the genetic algorithms with artificial neural networks for optimization of board allocating

This paper introduced the Genetic Algorithms (GAs) and Artificial Neural Networks (ANNs), which have been widely used in optimization of allocating. The combination way of the two optimizing algorithms was used in board allocating of furniture production. In the experiment, the rectangular flake board of 3650 mm 1850 mm was used as raw material to allocate 100 sets of Table Bucked. The utilizing rate of the board reached 94.14 % and the calculating time was only 35 s. The experiment result proofed that the method by using the GA for optimizing the weights of the ANN can raise the utilizing rate of the board and can shorten the time of the design. At the same time, this method can simultaneously searched in many directions, thus greatly in-creasing the probability of finding a global optimum.

A Four-color Matching Method Combining Neural Networks with Genetic Algorithm

A brief review of color matching technology and its application of printing RGB images by CMY or CMYK ink jet printers is presented, followed by an explanation to the conventional approaches that are commonly used in color matching. Then, a four color matching method combining neural network with genetic algorithm is proposed. The initial weights and thresholds of the BP neural network for RGB to CMY color conversion are optimized by the new genetic algorithm based on evolutionarily stable strategy. The fourth component K is generated by using GCR (Gray Component Replacement) concept. Simulation experiments show that it is well behaved in both accuracy and generalization performance.

Proton exchange membrane fuel cells modeling based on artificial neural networks

To understand the complexity of the mathematical models of a proton exchange membrane fuel cell (PEMFC) and their shortage of practical PEMFC control, the PEMFC complex mechanism and the existing PEMFC models are analyzed, and artificial neural networks based PEMFC modeling is advanced. The structure, algorithm, training and simulation of PEMFC modeling based on improved BP networks are given out in detail. The computer simulation and conducted experiment verify that this model is fast and accurate, and can be used as a suitable operational model for PEMFC real-time control.

Fast Learning in Spiking Neural Networks by Learning Rate Adaptation

For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.

ANN Model and Learning Algorithm in Fault Diagnosis for FMS

The fault diagnosis model for FMS based on multi layer feedforward neural networks was discussed An improved BP algorithm,the tactic of initial value selection based on genetic algorithm and the method of network structure optimization were presented for training this model ANN(artificial neural network)fault diagnosis model for the robot in FMS was made by the new algorithm The result is superior to the rtaditional algorithm

Modeling of mechanical properties of as-cast Mg-Li-Al alloys based on PSO-BP algorithm

Artificial neural networks have been widely used to predict the mechanical properties of alloys in material research. This study aims to investigate the implicit relationship between the compositions and mechanical properties of as-cast Mg-Li-AI alloys. Based on the experimental collection of the tensile strength and the elongation of representative Mg-Li-AI alloys, a momentum back-propagation （BP） neural network with a single hidden layer was established. Particle swarm optimization （PSO） was applied to optimize the BP model. In the neural network, the input variables were the contents of Mg, Li and AI, and the output variables were the tensile strength and the elongation. The results show that the proposed PSO-BP model can describe the quantitative relationship between the Mg-Li-AI alloy＇s composition and its mechanical properties. It is possible that the mechanical properties to be predicted without experiment by inputting the alloy composition into the trained network model. The prediction of the influence of AI addition on the mechanical properties of as-cast Mg-Li-AI alloys is consistent with the related research results.

Neural Network Based on GA-BP Algorithm and its Application in the Protein Secondary Structure Prediction

The advantages and disadvantages of genetic algorithm and BP algorithm are introduced. A neural network based on GA-BP algorithm is proposed and applied in the prediction of protein secondary structure, which combines the advantages of BP and GA. The prediction and training on the neural network are made respectively based on 4 structure classifications of protein so as to get higher rate of predication---the highest prediction rate 75.65%,the average prediction rate 65.04%.

Extended Deep Learning Algorithm for Improved Brain Tumor Diagnosis System

At present,the prediction of brain tumors is performed using Machine Learning(ML)and Deep Learning(DL)algorithms.Although various ML and DL algorithms are adapted to predict brain tumors to some range,some concerns still need enhancement,particularly accuracy,sensitivity,false positive and false negative,to improve the brain tumor prediction system symmetrically.Therefore,this work proposed an Extended Deep Learning Algorithm(EDLA)to measure performance parameters such as accuracy,sensitivity,and false positive and false negative rates.In addition,these iterated measures were analyzed by comparing the EDLA method with the Convolutional Neural Network(CNN)way further using the SPSS tool,and respective graphical illustrations were shown.The results were that the mean performance measures for the proposed EDLA algorithm were calculated,and those measured were accuracy(97.665%),sensitivity(97.939%),false positive(3.012%),and false negative(3.182%)for ten iterations.Whereas in the case of the CNN,the algorithm means accuracy gained was 94.287%,mean sensitivity 95.612%,mean false positive 5.328%,and mean false negative 4.756%.These results show that the proposed EDLA method has outperformed existing algorithms,including CNN,and ensures symmetrically improved parameters.Thus EDLA algorithm introduces novelty concerning its performance and particular activation function.This proposed method will be utilized effectively in brain tumor detection in a precise and accurate manner.This algorithm would apply to brain tumor diagnosis and be involved in various medical diagnoses aftermodification.If the quantity of dataset records is enormous,then themethod’s computation power has to be updated.

Assessing the Forecasting of Comprehensive Loss Incurred by Typhoons:A Combined PCA and BP Neural Network Model

This paper develops a joint model utilizing the principal component analysis(PCA)and the back propagation(BP)neural network model optimized by the Levenberg Marquardt(LM)algorithm,and as an application of the joint model to investigate the damages caused by typhoons for a coastal province,Fujian Province,China in 2005-2015(latest).First,the PCA is applied to analyze comprehensively the relationship between hazard factors,hazard bearing factors and disaster factors.Then five integrated indices,overall disaster level,typhoon intensity,damaged condition of houses,medical rescue and self-rescue capability,are extracted through the PCA;Finally,the BP neural network model,which takes the principal component scores as input and is optimized by the LM algorithm,is implemented to forecast the comprehensive loss of typhoons.It is estimated that an average annual loss of 138.514 billion RMB occurred for 2005-2015,with a maximum loss of 215.582 in 2006 and a decreasing trend since 2010 though the typhoon intensity increases.The model was validated using three typhoon events and it is found that the error is less than 1%.These results provide information for the government to increase medical institutions and medical workers and for the communities to promote residents’self-rescue capability.

NONLINEAR MODELING AND CONTROLLING OF ARTIFICIAL MUSCLE SYSTEM USING NEURAL NETWORKS

The pneumatic artificial muscles are widely used in the fields of medicalrobots, etc. Neural networks are applied to modeling and controlling of artificial muscle system. Asingle-joint artificial muscle test system is designed. The recursive prediction error (RPE)algorithm which yields faster convergence than back propagation (BP) algorithm is applied to trainthe neural networks. The realization of RPE algorithm is given. The difference of modeling ofartificial muscles using neural networks with different input nodes and different hidden layer nodesis discussed. On this basis the nonlinear control scheme using neural networks for artificialmuscle system has been introduced. The experimental results show that the nonlinear control schemeyields faster response and higher control accuracy than the traditional linear control scheme.

Integration of Learning Algorithm on Fuzzy Min-Max Neural Networks

An integrated fuzzy min-max neural network(IFMMNN) is developed to avoid the classification result influenced by the input sequence of training samples, and the learning algorithm can be used as pure clustering,pure classification, or a hybrid clustering classification. Three experiments are designed to realize the aim. The serial input of samples is changed to parallel input, and the fuzzy membership function is substituted by similarity matrix. The experimental results show its superiority in contrast with the original method proposed by Simpson.