CNN-LSTM based incremental attention mechanism enabled phase-space reconstruction for chaotic time series prediction

To improve the prediction accuracy of chaotic time series and reconstruct a more reasonable phase space structure of the prediction network,we propose a convolutional neural network-long short-term memory(CNN-LSTM)prediction model based on the incremental attention mechanism.Firstly,a traversal search is conducted through the traversal layer for finite parameters in the phase space.Then,an incremental attention layer is utilized for parameter judgment based on the dimension weight criteria(DWC).The phase space parameters that best meet DWC are selected and fed into the input layer.Finally,the constructed CNN-LSTM network extracts spatio-temporal features and provides the final prediction results.The model is verified using Logistic,Lorenz,and sunspot chaotic time series,and the performance is compared from the two dimensions of prediction accuracy and network phase space structure.Additionally,the CNN-LSTM network based on incremental attention is compared with long short-term memory(LSTM),convolutional neural network(CNN),recurrent neural network(RNN),and support vector regression(SVR)for prediction accuracy.The experiment results indicate that the proposed composite network model possesses enhanced capability in extracting temporal features and achieves higher prediction accuracy.Also,the algorithm to estimate the phase space parameter is compared with the traditional CAO,false nearest neighbor,and C-C,three typical methods for determining the chaotic phase space parameters.The experiments reveal that the phase space parameter estimation algorithm based on the incremental attention mechanism is superior in prediction accuracy compared with the traditional phase space reconstruction method in five networks,including CNN-LSTM,LSTM,CNN,RNN,and SVR.

A prediction comparison between univariate and multivariate chaotic time series

The methods to determine time delays and embedding dimensions in the phase space delay reconstruction of multivariate chaotic time series are proposed. Three nonlinear prediction methods of multivariate chaotic time series including local mean prediction, local linear prediction and BP neural networks prediction are considered. The simulation results obtained by the Lorenz system show that no matter what nonlinear prediction method is used, the prediction error of multivariate chaotic time series is much smaller than the prediction error of univariate time series, even if half of the data of univariate time series are used in multivariate time series. The results also verify that methods to determine the time delays and the embedding dimensions are correct from the view of minimizing the prediction error.

Chaotic time series multi-step direct prediction with partial least squares regression

Considering chaotic time series multi-step prediction, multi-step direct prediction model based on partial least squares （PLS） is proposed in this article, where PLS, the method for predicting a set of dependent variables forming a large set of predictors, is used to model the dynamic evolution between the space points and the corresponding future points. The model can eliminate error accumulation with the common single-step local model algorithm~ and refrain from the high multi-collinearity problem in the reconstructed state space with the increase of embedding dimension. Simulation predictions are done on the Mackey-Glass chaotic time series with the model. The satisfying prediction accuracy is obtained and the model efficiency verified. In the experiments, the number of extracted components in PLS is set with cross-validation procedure.

Local polynomial prediction method of multivariate chaotic time series and its application

To improve the prediction accuracy of chaotic time series, a new methodformed on the basis of local polynomial prediction is proposed. The multivariate phase spacereconstruction theory is utilized to reconstruct the phase space firstly, and on its basis, apolynomial function is applied to construct the prediction model, then the parameters of the modelaccording to the data matrix built with the embedding dimensions are estimated and a one-stepprediction value is calculated. An estimate and one-step prediction value is calculated. Finally,the mean squared root statistics are used to estimate the prediction effect. The simulation resultsobtained by the Lorenz system and the prediction results of the Shanghai composite index show thatthe local polynomial prediction errors of the multivariate chaotic time series are small and itsprediction accuracy is much higher than that of the univariate chaotic time series.

New prediction of chaotic time series based on local Lyapunov exponent

A new method of predicting chaotic time series is presented based on a local Lyapunov exponent, by quantitatively measuring the exponential rate of separation or attraction of two infinitely close trajectories in state space. After recon- structing state space from one-dimensional chaotic time series, neighboring multiple-state vectors of the predicting point are selected to deduce the prediction formula by using the definition of the locaI Lyapunov exponent. Numerical simulations are carded out to test its effectiveness and verify its higher precision over two older methods. The effects of the number of referential state vectors and added noise on forecasting accuracy are also studied numerically.

Multi-step-prediction of chaotic time series based on co-evolutionary recurrent neural network

This paper proposes a co-evolutionary recurrent neural network （CERNN） for the multi-step-prediction of chaotic time series, it estimates the proper parameters of phase space reconstruction and optimizes the structure of recurrent neural networks by coevolutionary strategy. The searching space was separated into two subspaces and the individuals are trained in a parallel computational procedure. It can dynamically combine the embedding method with the capability of recurrent neural network to incorporate past experience due to internal recurrence. The effectiveness of CERNN is evaluated by using three benchmark chaotic time series data sets： the Lorenz series, Mackey-Glass series and real-world sun spot series. The simulation results show that CERNN improves the performances of multi-step-prediction of chaotic time series.

Regular nonlinear response of the driven Duffng oscillator to chaotic time series

Nonlinear response of the driven Duffing oscillator to periodic or quasi-periodic signals has been well studied. In this paper, we investigate the nonlinear response of the driven Duffing oscillator to non-periodic, more specifically, chaotic time series. Through numerical simulations, we find that the driven Duffing oscillator can also show regular nonlinear response to the chaotic time series with different degree of chaos as generated by the same chaotic series generating model, and there exists a relationship between the state of the driven Duffing oscillator and the chaoticity of the input signal of the driven Duffing oscillator. One real-world and two artificial chaotic time series are used to verify the new feature of Duffing oscillator. A potential application of the new feature of Duffing oscillator is also indicated.

Neural Volterra filter for chaotic time series prediction

A new second-order neural Volterra filter （SONVF） with conjugate gradient （CG） algorithm is proposed to predict chaotic time series based on phase space delay-coordinate reconstruction of chaotic dynamics system in this paper, where the neuron activation functions are introduced to constraint Volterra series terms for improving the nonlinear approximation of second-order Volterra filter （SOVF）. The SONVF with CG algorithm improves the accuracy of prediction without increasing the computation complexity. Meanwhile, the difficulty of neuron number determination does not exist here. Experimental results show that the proposed filter can predict chaotic time series effectively, and one-step and multi-step prediction performances are obviously superior to those of SOVF, which demonstrate that the proposed SONVF is feasible and effective.

Chaotic time series prediction using fuzzy sigmoid kernel-based support vector machines

Support vector machines （SVM） have been widely used in chaotic time series predictions in recent years. In order to enhance the prediction efficiency of this method and implement it in hardware, the sigmoid kernel in SVM is drawn in a more natural way by using the fuzzy logic method proposed in this paper. This method provides easy hardware implementation and straightforward interpretability. Experiments on two typical chaotic time series predictions have been carried out and the obtained results show that the average CPU time can be reduced significantly at the cost of a small decrease in prediction accuracy, which is favourable for the hardware implementation for chaotic time series prediction.

The improved local linear prediction of chaotic time series

Based on the Bayesian information criterion, this paper proposes the improved local linear prediction method to predict chaotic time series. This method uses spatial correlation and temporal correlation simultaneously. Simulation results show that the improved local linear prediction method can effectively make multi-step and one-step prediction of chaotic time series and the multi-step prediction performance and one-step prediction accuracy of the improved local linear prediction method are superior to those of the traditional local linear prediction method.

Phase Space Prediction of Chaotic Time Series with Nu-Support Vector Machine Regression

A new class of support vector machine, nil-support vector machine, isdiscussed which can handle both classification and regression. We focus on nu-support vector machineregression and use it for phase space prediction of chaotic time series. The effectiveness of themethod is demonstrated by applying it to the Henon map. This study also compares nu-support vectormachine with back propagation (BP) networks in order to better evaluate the performance of theproposed methods. The experimental results show that the nu-support vector machine regressionobtains lower root mean squared error than the BP networks and provides an accurate chaotic timeseries prediction. These results can be attributable to the fact that nu-support vector machineimplements the structural risk minimization principle and this leads to better generalization thanthe BP networks.

PREDICTION TECHNIQUES OF CHAOTIC TIME SERIES AND ITS APPLICATIONS AT LOW NOISE LEVEL

The paper not only studies the noise reduction methods of chaotic time series with noise and its reconstruction techniques, but also discusses prediction techniques of chaotic time series and its applications based on chaotic data noise reduction. In the paper, we first decompose the phase space of chaotic time series to range space and null noise space. Secondly we restructure original chaotic time series in range space. Lastly on the basis of the above, we establish order of the nonlinear model and make use of the nonlinear model to predict some research. The result indicates that the nonlinear model has very strong ability of approximation function, and Chaos predict method has certain tutorial significance to the practical problems.

Stabilization of Chaotic Time Series by Proportional Pulse in the System Variable Based on Genetic Algorithm

The PPSV (Proportional Pulse in the System Variable) algorithm is a convenient method for the stabilization of the chaotic time series. It does not require any previous knowledge of the system. The PPSV method also has a shortcoming, that is, the determination off. is a procedure by trial and error, since it lacks of optimization. In order to overcome the blindness, GA (Genetic Algorithm), a search algorithm based on the mechanics of natural selection and natural genetics, is used to optimize the λi The new method is named as GAPPSV algorithm. The simulation results show that GAPPSV algorithm is very efficient because the control process is short and the steady-state error is small.

Multimodality Prediction of Chaotic Time Series with Sparse Hard-Cut EM Learning of the Gaussian Process Mixture Model

The contribution of this work is twofold： （1） a multimodality prediction method of chaotic time series with the Gaussian process mixture （GPM） model is proposed, which employs a divide and conquer strategy. It automatically divides the chaotic time series into multiple modalities with different extrinsic patterns and intrinsic characteristics, and thus can more precisely fit the chaotic time series. （2） An effective sparse hard-cut expec- tation maximization （SHC-EM） learning algorithm for the GPM model is proposed to improve the prediction performance. SHO-EM replaces a large learning sample set with fewer pseudo inputs, accelerating model learning based on these pseudo inputs. Experiments on Lorenz and Chua time series demonstrate that the proposed method yields not only accurate multimodality prediction, but also the prediction confidence interval SHC-EM outperforms the traditional variational 1earning in terms of both prediction accuracy and speed. In addition, SHC-EM is more robust and insusceptible to noise than variational learning.

Genetic programming-based chaotic time series modeling

This paper proposes a Genetic Programming-Based Modeling (GPM) algorithm on chaotic time series. GP is used here to search for appropriate model structures in function space, and the Particle Swarm Optimization (PSO) algorithm is used for Nonlinear Parameter Estimation (NPE) of dynamic model structures. In addition, GPM integrates the results of Nonlinear Time Series Analysis (NTSA) to adjust the parameters and takes them as the criteria of established models. Experiments showed the effectiveness of such improvements on chaotic time series modeling.

Complex Networks from Chaotic Time Series on Riemannian Manifold

Complex networks are important paradigms for analyzing the complex systems as they allow understanding the structural properties of systems composed of different interacting entities. In this work we propose a reliable method for constructing complex networks from chaotic time series. We first estimate the covariance matrices, then a geodesic-based distance between the covariance matrices is introduced. Consequently the network can be constructed on a Riemannian manifold where the nodes and edges correspond to the covariance matrix and geodesic-based distance, respectively. The proposed method provides us with an intrinsic geometry viewpoint to understand the time series.

Prediction and analysis of chaotic time series on the basis of support vector

Based on discussion on the theories of support vector machines （SVM）, an one-step prediction model for time series prediction is presented, wherein the chaos theory is incorporated. Chaotic character of the time series is taken into account in the prediction procedure; parameters of reconstruction-detay and embedding-dimension for phase-space reconstruction are calculated in light of mutual-information and false-nearest-neighbor method, respectively. Precision and functionality have been demonstrated by the experimental results on the basis of the prediction of Lorenz chaotic time series.

A method to improve the precision of chaotic time series prediction by using a non-trajectory

Due to the error in the measured value of the initial state and the sensitive dependence on initial conditions of chaotic dynamical systems, the error of chaotic time series prediction increases with the prediction step. This paper provides a method to improve the prediction precision by adjusting the predicted value in the course of iteration according to the evolution information of small intervals on the left and right sides of the predicted value. The adjusted predicted result is a non-trajectory which can provide a better prediction performance than the usual result based on the trajectory. Numerical simulations of two typical chaotic maps demonstrate its effectiveness. When the prediction step gets relatively larger, the effect is more pronounced.

KLT-based local linear prediction of chaotic time series

In the reconstructed phase space, based on the Karhunen-Loeve transformation （KLT）, the new local linear prediction method is proposed to predict chaotic time series. ＆ noise-free chaotic time series and a noise added chaotic time series are analyzed. The simulation results show that the KLT-based local linear prediction method can effectively make one-step and multi-step prediction for chaotic time series, and the one-step and multi-step prediction accuracies of the KLT-based local linear prediction method are superior to that of the traditional local linear prediction.

Research on Optimize Prediction Model and Algorithm about Chaotic Time Series

We put forward a chaotic estimating model, by using the parameter of the chaotic system, sensitivity of the parameter to inching and control the disturbance of the system, and estimated the parameter of the model by using the best update option. In the end, we forecast the intending series value in its mutually space. The example shows that it can increase the precision in the estimated process by selecting the best model steps. It not only conquer the abuse of using detention inlay technology alone, but also decrease blindness of using forecast error to decide the input model directly, and the result of it is better than the method of statistics and other series means. Key words chaotic time series - parameter identification - optimal prediction model - improved change ruler method CLC number TP 273 Foundation item: Supported by the National Natural Science Foundation of China (60373062)Biography: JIANG Wei-jin (1964-), male, Professor, research direction: intelligent compute and the theory methods of distributed data processing in complex system, and the theory of software.