A novel adaptive support vector regression neural network （SVR-NN） is proposed, which combines respectively merits of support vector machines and a neural network. First, a support vector regression approach is appl...A novel adaptive support vector regression neural network （SVR-NN） is proposed, which combines respectively merits of support vector machines and a neural network. First, a support vector regression approach is applied to determine the initial structure and initial weights of the SVR-NN so that the network architecture is easily determined and the hidden nodes can adaptively be constructed based on support vectors. Furthermore, an annealing robust learning algorithm is presented to adjust these hidden node parameters as well as the weights of the SVR-NN. To test the validity of the proposed method, it is demonstrated that the adaptive SVR-NN can be used effectively for the identification of nonlinear dynamic systems. Simulation results show that the identification schemes based on the SVR-NN give considerably better performance and show faster learning in comparison to the previous neural network method.展开更多
The adaptive learning and prediction of a highly nonlinear and time-varying bioreactor benchmark process is studied using Neur-On-Line, a graphical tool kit for developing and deploying neural networks in the G2 real ...The adaptive learning and prediction of a highly nonlinear and time-varying bioreactor benchmark process is studied using Neur-On-Line, a graphical tool kit for developing and deploying neural networks in the G2 real time intelligent environment,and a new modified Broyden, Fletcher, Goldfarb, and Shanno （BFGS） quasi-Newton algorithm. The modified BFGS algorithm for the adaptive learning of back propagation （BP） neural networks is developed and embedded into NeurOn-Line by introducing a new search method of learning rate to the full memory BFGS algorithm. Simulation results show that the adaptive learning and prediction neural network system can quicklv track the time-varving and nonlinear behavior of the bioreactor.展开更多
For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and de...For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.展开更多
A self-adaptive learning based immune algorithm （SALIA） is proposed to tackle diverse optimization problems, such as complex multi-modal and ill-conditioned prc,blems with the high robustness. The SALIA algorithm ad...A self-adaptive learning based immune algorithm （SALIA） is proposed to tackle diverse optimization problems, such as complex multi-modal and ill-conditioned prc,blems with the high robustness. The SALIA algorithm adopted a mutation strategy pool which consists of four effective mutation strategies to generate new antibodies. A self-adaptive learning framework is implemented to select the mutation strategies by learning from their previous performances in generating promising solutions. Twenty-six state-of-the-art optimization problems with different characteristics, such as uni-modality, multi-modality, rotation, ill-condition, mis-scale and noise, are used to verify the validity of SALIA. Experimental results show that the novel algorithm SALIA achieves a higher universality and robustness than clonal selection algorithms （CLONALG）, and the mean error index of each test function in SALIA decreases by a factor of at least 1.0×10^7 in average.展开更多
This paper focuses on resolving the identification problem of a neuro-fuzzy model(NFM) applied in batch processes. A hybrid learning algorithm is introduced to identify the proposed NFM with the idea of auxiliary erro...This paper focuses on resolving the identification problem of a neuro-fuzzy model(NFM) applied in batch processes. A hybrid learning algorithm is introduced to identify the proposed NFM with the idea of auxiliary error model and the identification principle based on the probability density function(PDF). The main contribution is that the NFM parameter updating approach is transformed into the shape control for the PDF of modeling error. More specifically, a virtual adaptive control system is constructed with the aid of the auxiliary error model and then the PDF shape control idea is used to tune NFM parameters so that the PDF of modeling error is controlled to follow a targeted PDF, which is in Gaussian or uniform distribution. Examples are used to validate the applicability of the proposed method and comparisons are made with the minimum mean square error based approaches.展开更多
文摘A novel adaptive support vector regression neural network （SVR-NN） is proposed, which combines respectively merits of support vector machines and a neural network. First, a support vector regression approach is applied to determine the initial structure and initial weights of the SVR-NN so that the network architecture is easily determined and the hidden nodes can adaptively be constructed based on support vectors. Furthermore, an annealing robust learning algorithm is presented to adjust these hidden node parameters as well as the weights of the SVR-NN. To test the validity of the proposed method, it is demonstrated that the adaptive SVR-NN can be used effectively for the identification of nonlinear dynamic systems. Simulation results show that the identification schemes based on the SVR-NN give considerably better performance and show faster learning in comparison to the previous neural network method.
文摘The adaptive learning and prediction of a highly nonlinear and time-varying bioreactor benchmark process is studied using Neur-On-Line, a graphical tool kit for developing and deploying neural networks in the G2 real time intelligent environment,and a new modified Broyden, Fletcher, Goldfarb, and Shanno （BFGS） quasi-Newton algorithm. The modified BFGS algorithm for the adaptive learning of back propagation （BP） neural networks is developed and embedded into NeurOn-Line by introducing a new search method of learning rate to the full memory BFGS algorithm. Simulation results show that the adaptive learning and prediction neural network system can quicklv track the time-varving and nonlinear behavior of the bioreactor.
基金Supported by the National Natural Science Foundation of China (60904018, 61203040)the Natural Science Foundation of Fujian Province of China (2009J05147, 2011J01352)+1 种基金the Foundation for Distinguished Young Scholars of Higher Education of Fujian Province of China (JA10004)the Science Research Foundation of Huaqiao University (09BS617)
文摘For accelerating the supervised learning by the SpikeProp algorithm with the temporal coding paradigm in spiking neural networks (SNNs), three learning rate adaptation methods (heuristic rule, delta-delta rule, and delta-bar-delta rule), which are used to speed up training in artificial neural networks, are used to develop the training algorithms for feedforward SNN. The performance of these algorithms is investigated by four experiments: classical XOR (exclusive or) problem, Iris dataset, fault diagnosis in the Tennessee Eastman process, and Poisson trains of discrete spikes. The results demonstrate that all the three learning rate adaptation methods are able to speed up convergence of SNN compared with the original SpikeProp algorithm. Furthermore, if the adaptive learning rate is used in combination with the momentum term, the two modifications will balance each other in a beneficial way to accomplish rapid and steady convergence. In the three learning rate adaptation methods, delta-bar-delta rule performs the best. The delta-bar-delta method with momentum has the fastest convergence rate, the greatest stability of training process, and the maximum accuracy of network learning. The proposed algorithms in this paper are simple and efficient, and consequently valuable for practical applications of SNN.
基金Project(2010ZC13012) supported by the Aviation Science Funds of China
文摘A self-adaptive learning based immune algorithm （SALIA） is proposed to tackle diverse optimization problems, such as complex multi-modal and ill-conditioned prc,blems with the high robustness. The SALIA algorithm adopted a mutation strategy pool which consists of four effective mutation strategies to generate new antibodies. A self-adaptive learning framework is implemented to select the mutation strategies by learning from their previous performances in generating promising solutions. Twenty-six state-of-the-art optimization problems with different characteristics, such as uni-modality, multi-modality, rotation, ill-condition, mis-scale and noise, are used to verify the validity of SALIA. Experimental results show that the novel algorithm SALIA achieves a higher universality and robustness than clonal selection algorithms （CLONALG）, and the mean error index of each test function in SALIA decreases by a factor of at least 1.0×10^7 in average.
基金Supported by the National Natural Science Foundation of China(61374044)Shanghai Science Technology Commission(12510709400)+1 种基金Shanghai Municipal Education Commission(14ZZ088)Shanghai Talent Development Plan
文摘This paper focuses on resolving the identification problem of a neuro-fuzzy model(NFM) applied in batch processes. A hybrid learning algorithm is introduced to identify the proposed NFM with the idea of auxiliary error model and the identification principle based on the probability density function(PDF). The main contribution is that the NFM parameter updating approach is transformed into the shape control for the PDF of modeling error. More specifically, a virtual adaptive control system is constructed with the aid of the auxiliary error model and then the PDF shape control idea is used to tune NFM parameters so that the PDF of modeling error is controlled to follow a targeted PDF, which is in Gaussian or uniform distribution. Examples are used to validate the applicability of the proposed method and comparisons are made with the minimum mean square error based approaches.
