A new type of recurrent neural network is discussed, which provides the potential for modelling unknown nonlinear systems. The proposed network is a generalization of the network described by Elman, which has three la...A new type of recurrent neural network is discussed, which provides the potential for modelling unknown nonlinear systems. The proposed network is a generalization of the network described by Elman, which has three layers including the input layer, the hidden layer and the output layer. The input layer is composed of two different groups of neurons, the group of external input neurons and the group of the internal context neurons. Since arbitrary connections can be allowed from the hidden layer to the context layer, the modified Elman network has more memory space to represent dynamic systems than the Elman network. In addition, it is proved that the proposed network with appropriate neurons in the context layer can approximate the trajectory of a given dynamical system for any fixed finite length of time. The dynamic backpropagation algorithm is used to estimate the weights of both the feedforward and feedback connections. The methods have been successfully applied to the modelling of nonlinear plants.展开更多
In order to enhance the accuracy and reliability of wireless location under non-line-of-sight (NLOS) environments,a novel neural network (NN) location approach using the digital broadcasting signals is presented. ...In order to enhance the accuracy and reliability of wireless location under non-line-of-sight (NLOS) environments,a novel neural network (NN) location approach using the digital broadcasting signals is presented. By the learning ability of the NN and the closely approximate unknown function to any degree of desired accuracy,the input-output mapping relationship between coordinates and the measurement data of time of arrival (TOA) and time difference of arrival (TDOA) is established. A real-time learning algorithm based on the extended Kalman filter (EKF) is used to train the multilayer perceptron (MLP) network by treating the linkweights of a network as the states of the nonlinear dynamic system. Since the EKF-based learning algorithm approximately gives the minimum variance estimate of the linkweights,the convergence is improved in comparison with the backwards error propagation (BP) algorithm. Numerical results illustrate thatthe proposedalgorithmcanachieve enhanced accuracy,and the performance ofthe algorithmis betterthanthat of the BP-based NN algorithm and the least squares (LS) algorithm in the NLOS environments. Moreover,this location method does not depend on a particular distribution of the NLOS error and does not need line-of-sight ( LOS ) or NLOS identification.展开更多
This letter analyzes the reasons why the known Neural Back Promulgation (NBP)network learning algorithm has slower speed and greater sample error. Based on the analysis and experiment, the training group descending En...This letter analyzes the reasons why the known Neural Back Promulgation (NBP)network learning algorithm has slower speed and greater sample error. Based on the analysis and experiment, the training group descending Enhanced Combination Algorithm (ECA) is proposed.The analysis of the generalized property and sample error shows that the ECA can heighten the study speed and reduce individual error.展开更多
The factors that influence underwater vehicle sonar self-noise are analyzed, and genetic algorithms and a back propagation (BP) neural network are combined to predict underwater vehicle sonar self-noise. The experimen...The factors that influence underwater vehicle sonar self-noise are analyzed, and genetic algorithms and a back propagation (BP) neural network are combined to predict underwater vehicle sonar self-noise. The experimental results demonstrate that underwater vehicle sonar self-noise can be predicted accurately by a GA-BP neural network that is based on actual underwater vehicle sonar data.展开更多
In the context of new risks and threats associated to nuclear, biological and chemical (NBC) attacks, and given the shortcomings of certain analytical methods such as principal component analysis (PCA), a neural n...In the context of new risks and threats associated to nuclear, biological and chemical (NBC) attacks, and given the shortcomings of certain analytical methods such as principal component analysis (PCA), a neural network approach seems to be more accurate. PCA consists in projecting the spectrum of a gas collected from a remote sensing system in, firstly, a three-dimensional space, then in a two-dimensional one using a model of Multi-Layer Perceptron based neural network. It adopts during the learning process, the back propagation algorithm of the gradient, in which the mean square error output is continuously calculated and compared to the input until it reaches a minimal threshold value. This aims to correct the synaptic weights of the network. So, the Artificial Neural Network (ANN) tends to be more efficient in the classification process. This paper emphasizes the contribution of the ANN method in the spectral data processing, classification and identification and in addition, its fast convergence during the back propagation of the gradient.展开更多
Based on the structure of Elman and Jordan neural networks, a new dynamic neural network is constructed. The network can remember the past state of the hidden layer and adjust the effect of the past signal to the curr...Based on the structure of Elman and Jordan neural networks, a new dynamic neural network is constructed. The network can remember the past state of the hidden layer and adjust the effect of the past signal to the current value in real-time. And in order to enhance the signal processing capabilities, the feedback of output layer nodes is increased. A hybrid learning algorithm based on genetic algorithm (GA) and error back propagation algorithm (BP) is used to adjust the weight values of the network, which can accelerate the rate of convergence and avoid getting into local optimum. Finally, the improved neural network is utilized to identify underwater vehicle (UV) ' s hydrodynamic model, and the simulation results show that the neural network based on hybrid learning algorithm can improve the learning rate of convergence and identification nrecision.展开更多
The authors will examine prediction of temperature daily profile using various modifications of BPTT (backpropagation through time algorithm) done by stochastic update in the artificial RCNN (recurrent neural netwo...The authors will examine prediction of temperature daily profile using various modifications of BPTT (backpropagation through time algorithm) done by stochastic update in the artificial RCNN (recurrent neural networks). The general introduction was provided by Salvetti and Wilamowski in 1994 in order to improve probability of convergence and speed of convergence. This update method has also one another quality, its implementation is simple for arbitrary network topology. In stochastic update scenario, constant number of weights/neurons is randomly selected and updated. This is in contrast to classical ordered update, where always all weights/neurons are updated. Stochastic update is suitable to replace classical ordered update without any penalty on implementation complexity and with good chance without penalty on quality of convergence. They have provided first experiments with stochastic modification on BP (backpropagation algorithm) used for artificial FFNN (feed-forward neural network) in detail described in the article "Stochastic Weight Update in the Backpropagation Algorithm on Feed-Forward Neural Networks" presented on the conference IJCNN (International Joint Conference of Neural Networks) 2010 in Barcelona. The BPTT on RCNN uses the history of previous steps stored inside of the NN that can be used for prediction. They will describe exact implementation on the RCNN, and present experiment results on temperature prediction with recurrent neural network topology. The dataset used for temperature prediction consists of the measured temperature from the year 2000 till the end of February 2011. Dataset is split into two groups: training dataset, which is provided to network in learning phase, and testing dataset, which is unknown part of dataset to NN and used to test the ability of NN to predict the temperature and the ability of NN to generalize the model hidden in the temperature profile. The results show promising properties of stochastic weight update with toy-task data, and the higher complexity of the temperature daily profile prediction.展开更多
One of the main concerns in Engineering nowadays is the development of aircrafts of low consumption and high performance. For this purpose, airfoils are studied and designed to have an elevated lift coefficient and a ...One of the main concerns in Engineering nowadays is the development of aircrafts of low consumption and high performance. For this purpose, airfoils are studied and designed to have an elevated lift coefficient and a low drag coefficient, thus generating a highly efficient airfoil. The higher the efficiency value is, the lower the aircraft fuel consumption will be; thus improving its performance. In this sense, this work aims to develop a tool for airfoil creation from some desired characteristics, such as the lift and drag coefficients and maximum efficiency rate, using an algorithm based on an ANN (artificial neural network). In order to do so, a database of aerodynamic characteristics with a total of 300 airfoils was initially collected from the XFoil software. Then, through a routine implemented in the MATLAB software, network architectures of one, two, three and four modules were trained, using the back propagation algorithm and momentum. The cross-validation technique was applied to analyze the results, evaluating which network possesses the lowest value in RMS (root-mean-square) error. In this case, the best result obtained was from the two-module architecture with two hidden neuron layers. The airfoils developed by this network, in the regions with the lowest RMS, were compared to the same airfoils imported to XFoil. The presented work offers as a contribution, in relation to other works involving ANN applied to fluid mechanics, the development of airfoils from their aerodynamic characteristics.展开更多
The application of support vector machines to forecasting problems is becoming popular, lately. Several comparisons between neural networks trained with error backpropagation and support vector machines have shown adv...The application of support vector machines to forecasting problems is becoming popular, lately. Several comparisons between neural networks trained with error backpropagation and support vector machines have shown advantage for the latter in different domains of application. However, some difficulties still deteriorate the performance of the support vector machines. The main one is related to the setting of the hyperparameters involved in their training. Techniques based on meta-heuristics have been employed to determine appropriate values for those hyperparameters. However, because of the high noneonvexity of this estimation problem, which makes the search for a good solution very hard, an approach based on Bayesian inference, called relevance vector machine, has been proposed more recently. The present paper aims at investigating the suitability of this new approach to the short-term load forecasting problem.展开更多
An important problem in demand planning for energy consumption is developing an accurate energy forecasting model. In fact, it is not possible to allocate the energy resources in an optimal manner without having accur...An important problem in demand planning for energy consumption is developing an accurate energy forecasting model. In fact, it is not possible to allocate the energy resources in an optimal manner without having accurate demand value. A new energy forecasting model was proposed based on the back-propagation(BP) type neural network and imperialist competitive algorithm. The proposed method offers the advantage of local search ability of BP technique and global search ability of imperialist competitive algorithm. Two types of empirical data regarding the energy demand(gross domestic product(GDP), population, import, export and energy demand) in Turkey from 1979 to 2005 and electricity demand(population, GDP, total revenue from exporting industrial products and electricity consumption) in Thailand from 1986 to 2010 were investigated to demonstrate the applicability and merits of the present method. The performance of the proposed model is found to be better than that of conventional back-propagation neural network with low mean absolute error.展开更多
A new method for prediction of wing aerodynamic performance in rain condition was presented.Three-and four-layer artificial neural networks based on improved algorithm for error Back Propagation(BP)network were respec...A new method for prediction of wing aerodynamic performance in rain condition was presented.Three-and four-layer artificial neural networks based on improved algorithm for error Back Propagation(BP)network were respectively built.Detailed approaches to determine the optical parameters for network model were introduced and the specific steps for applying BP network model to predict wing aerodynamic performance in rain were given.On this basis,the established optimal three-and four-layer BP network model was used for this prediction.Results indicate that both of the network models are appropriate for predicting wing aerodynamic performance in rain.The sum of square error level produced by two models is less than 0.2%,and the prediction accuracy by four-layer network model is higher than that of three-layer network.展开更多
文摘A new type of recurrent neural network is discussed, which provides the potential for modelling unknown nonlinear systems. The proposed network is a generalization of the network described by Elman, which has three layers including the input layer, the hidden layer and the output layer. The input layer is composed of two different groups of neurons, the group of external input neurons and the group of the internal context neurons. Since arbitrary connections can be allowed from the hidden layer to the context layer, the modified Elman network has more memory space to represent dynamic systems than the Elman network. In addition, it is proved that the proposed network with appropriate neurons in the context layer can approximate the trajectory of a given dynamical system for any fixed finite length of time. The dynamic backpropagation algorithm is used to estimate the weights of both the feedforward and feedback connections. The methods have been successfully applied to the modelling of nonlinear plants.
基金The National High Technology Research and Development Program of China (863 Program) (No.2008AA01Z227)the Cultivatable Fund of the Key Scientific and Technical Innovation Project of Ministry of Education of China (No.706028)
文摘In order to enhance the accuracy and reliability of wireless location under non-line-of-sight (NLOS) environments,a novel neural network (NN) location approach using the digital broadcasting signals is presented. By the learning ability of the NN and the closely approximate unknown function to any degree of desired accuracy,the input-output mapping relationship between coordinates and the measurement data of time of arrival (TOA) and time difference of arrival (TDOA) is established. A real-time learning algorithm based on the extended Kalman filter (EKF) is used to train the multilayer perceptron (MLP) network by treating the linkweights of a network as the states of the nonlinear dynamic system. Since the EKF-based learning algorithm approximately gives the minimum variance estimate of the linkweights,the convergence is improved in comparison with the backwards error propagation (BP) algorithm. Numerical results illustrate thatthe proposedalgorithmcanachieve enhanced accuracy,and the performance ofthe algorithmis betterthanthat of the BP-based NN algorithm and the least squares (LS) algorithm in the NLOS environments. Moreover,this location method does not depend on a particular distribution of the NLOS error and does not need line-of-sight ( LOS ) or NLOS identification.
基金the National Defense Research item "Data fusion" of Tenth Five-Year Plan 102010203
文摘This letter analyzes the reasons why the known Neural Back Promulgation (NBP)network learning algorithm has slower speed and greater sample error. Based on the analysis and experiment, the training group descending Enhanced Combination Algorithm (ECA) is proposed.The analysis of the generalized property and sample error shows that the ECA can heighten the study speed and reduce individual error.
文摘The factors that influence underwater vehicle sonar self-noise are analyzed, and genetic algorithms and a back propagation (BP) neural network are combined to predict underwater vehicle sonar self-noise. The experimental results demonstrate that underwater vehicle sonar self-noise can be predicted accurately by a GA-BP neural network that is based on actual underwater vehicle sonar data.
文摘In the context of new risks and threats associated to nuclear, biological and chemical (NBC) attacks, and given the shortcomings of certain analytical methods such as principal component analysis (PCA), a neural network approach seems to be more accurate. PCA consists in projecting the spectrum of a gas collected from a remote sensing system in, firstly, a three-dimensional space, then in a two-dimensional one using a model of Multi-Layer Perceptron based neural network. It adopts during the learning process, the back propagation algorithm of the gradient, in which the mean square error output is continuously calculated and compared to the input until it reaches a minimal threshold value. This aims to correct the synaptic weights of the network. So, the Artificial Neural Network (ANN) tends to be more efficient in the classification process. This paper emphasizes the contribution of the ANN method in the spectral data processing, classification and identification and in addition, its fast convergence during the back propagation of the gradient.
基金Supported by the Postdoctoral Science Foundation of China( No. 20100480964 ) , the Basic Research Foundation of Central University ( No. HEUCF100104) and the National Natural Science Foundation of China (No. 50909025/E091002).
文摘Based on the structure of Elman and Jordan neural networks, a new dynamic neural network is constructed. The network can remember the past state of the hidden layer and adjust the effect of the past signal to the current value in real-time. And in order to enhance the signal processing capabilities, the feedback of output layer nodes is increased. A hybrid learning algorithm based on genetic algorithm (GA) and error back propagation algorithm (BP) is used to adjust the weight values of the network, which can accelerate the rate of convergence and avoid getting into local optimum. Finally, the improved neural network is utilized to identify underwater vehicle (UV) ' s hydrodynamic model, and the simulation results show that the neural network based on hybrid learning algorithm can improve the learning rate of convergence and identification nrecision.
文摘The authors will examine prediction of temperature daily profile using various modifications of BPTT (backpropagation through time algorithm) done by stochastic update in the artificial RCNN (recurrent neural networks). The general introduction was provided by Salvetti and Wilamowski in 1994 in order to improve probability of convergence and speed of convergence. This update method has also one another quality, its implementation is simple for arbitrary network topology. In stochastic update scenario, constant number of weights/neurons is randomly selected and updated. This is in contrast to classical ordered update, where always all weights/neurons are updated. Stochastic update is suitable to replace classical ordered update without any penalty on implementation complexity and with good chance without penalty on quality of convergence. They have provided first experiments with stochastic modification on BP (backpropagation algorithm) used for artificial FFNN (feed-forward neural network) in detail described in the article "Stochastic Weight Update in the Backpropagation Algorithm on Feed-Forward Neural Networks" presented on the conference IJCNN (International Joint Conference of Neural Networks) 2010 in Barcelona. The BPTT on RCNN uses the history of previous steps stored inside of the NN that can be used for prediction. They will describe exact implementation on the RCNN, and present experiment results on temperature prediction with recurrent neural network topology. The dataset used for temperature prediction consists of the measured temperature from the year 2000 till the end of February 2011. Dataset is split into two groups: training dataset, which is provided to network in learning phase, and testing dataset, which is unknown part of dataset to NN and used to test the ability of NN to predict the temperature and the ability of NN to generalize the model hidden in the temperature profile. The results show promising properties of stochastic weight update with toy-task data, and the higher complexity of the temperature daily profile prediction.
文摘One of the main concerns in Engineering nowadays is the development of aircrafts of low consumption and high performance. For this purpose, airfoils are studied and designed to have an elevated lift coefficient and a low drag coefficient, thus generating a highly efficient airfoil. The higher the efficiency value is, the lower the aircraft fuel consumption will be; thus improving its performance. In this sense, this work aims to develop a tool for airfoil creation from some desired characteristics, such as the lift and drag coefficients and maximum efficiency rate, using an algorithm based on an ANN (artificial neural network). In order to do so, a database of aerodynamic characteristics with a total of 300 airfoils was initially collected from the XFoil software. Then, through a routine implemented in the MATLAB software, network architectures of one, two, three and four modules were trained, using the back propagation algorithm and momentum. The cross-validation technique was applied to analyze the results, evaluating which network possesses the lowest value in RMS (root-mean-square) error. In this case, the best result obtained was from the two-module architecture with two hidden neuron layers. The airfoils developed by this network, in the regions with the lowest RMS, were compared to the same airfoils imported to XFoil. The presented work offers as a contribution, in relation to other works involving ANN applied to fluid mechanics, the development of airfoils from their aerodynamic characteristics.
文摘The application of support vector machines to forecasting problems is becoming popular, lately. Several comparisons between neural networks trained with error backpropagation and support vector machines have shown advantage for the latter in different domains of application. However, some difficulties still deteriorate the performance of the support vector machines. The main one is related to the setting of the hyperparameters involved in their training. Techniques based on meta-heuristics have been employed to determine appropriate values for those hyperparameters. However, because of the high noneonvexity of this estimation problem, which makes the search for a good solution very hard, an approach based on Bayesian inference, called relevance vector machine, has been proposed more recently. The present paper aims at investigating the suitability of this new approach to the short-term load forecasting problem.
文摘An important problem in demand planning for energy consumption is developing an accurate energy forecasting model. In fact, it is not possible to allocate the energy resources in an optimal manner without having accurate demand value. A new energy forecasting model was proposed based on the back-propagation(BP) type neural network and imperialist competitive algorithm. The proposed method offers the advantage of local search ability of BP technique and global search ability of imperialist competitive algorithm. Two types of empirical data regarding the energy demand(gross domestic product(GDP), population, import, export and energy demand) in Turkey from 1979 to 2005 and electricity demand(population, GDP, total revenue from exporting industrial products and electricity consumption) in Thailand from 1986 to 2010 were investigated to demonstrate the applicability and merits of the present method. The performance of the proposed model is found to be better than that of conventional back-propagation neural network with low mean absolute error.
文摘A new method for prediction of wing aerodynamic performance in rain condition was presented.Three-and four-layer artificial neural networks based on improved algorithm for error Back Propagation(BP)network were respectively built.Detailed approaches to determine the optical parameters for network model were introduced and the specific steps for applying BP network model to predict wing aerodynamic performance in rain were given.On this basis,the established optimal three-and four-layer BP network model was used for this prediction.Results indicate that both of the network models are appropriate for predicting wing aerodynamic performance in rain.The sum of square error level produced by two models is less than 0.2%,and the prediction accuracy by four-layer network model is higher than that of three-layer network.