Revisiting Akaike’s Final Prediction Error and the Generalized Cross Validation Criteria in Regression from the Same Perspective: From Least Squares to Ridge Regression and Smoothing Splines

In regression, despite being both aimed at estimating the Mean Squared Prediction Error (MSPE), Akaike’s Final Prediction Error (FPE) and the Generalized Cross Validation (GCV) selection criteria are usually derived from two quite different perspectives. Here, settling on the most commonly accepted definition of the MSPE as the expectation of the squared prediction error loss, we provide theoretical expressions for it, valid for any linear model (LM) fitter, be it under random or non random designs. Specializing these MSPE expressions for each of them, we are able to derive closed formulas of the MSPE for some of the most popular LM fitters: Ordinary Least Squares (OLS), with or without a full column rank design matrix;Ordinary and Generalized Ridge regression, the latter embedding smoothing splines fitting. For each of these LM fitters, we then deduce a computable estimate of the MSPE which turns out to coincide with Akaike’s FPE. Using a slight variation, we similarly get a class of MSPE estimates coinciding with the classical GCV formula for those same LM fitters.

Least Squares Evaluations for Form and Profile Errors of Ellipse Using Coordinate Data

To improve the measurement and evaluation of form error of an elliptic section, an evaluation method based on least squares fitting is investigated to analyze the form and profile errors of an ellipse using coordinate data. Two error indicators for defining ellipticity are discussed, namely the form error and the profile error, and the difference between both is considered as the main parameter for evaluating machining quality of surface and profile. Because the form error and the profile error rely on different evaluation benchmarks, the major axis and the foci rather than the centre of an ellipse are used as the evaluation benchmarks and can accurately evaluate a tolerance range with the separated form error and profile error of workpiece. Additionally, an evaluation program based on the LS model is developed to extract the form error and the profile error of the elliptic section, which is well suited for separating the two errors by a standard program. Finally, the evaluation method about the form and profile errors of the ellipse is applied to the measurement of skirt line of the piston, and results indicate the effectiveness of the evaluation. This approach provides the new evaluation indicators for the measurement of form and profile errors of ellipse, which is found to have better accuracy and can thus be used to solve the difficult of the measurement and evaluation of the piston in industrial production.

Penalized total least squares method for dealing with systematic errors in partial EIV model and its precision estimation

When the total least squares(TLS)solution is used to solve the parameters in the errors-in-variables(EIV)model,the obtained parameter estimations will be unreliable in the observations containing systematic errors.To solve this problem,we propose to add the nonparametric part(systematic errors)to the partial EIV model,and build the partial EIV model to weaken the influence of systematic errors.Then,having rewritten the model as a nonlinear model,we derive the formula of parameter estimations based on the penalized total least squares criterion.Furthermore,based on the second-order approximation method of precision estimation,we derive the second-order bias and covariance of parameter estimations and calculate the mean square error(MSE).Aiming at the selection of the smoothing factor,we propose to use the U curve method.The experiments show that the proposed method can mitigate the influence of systematic errors to a certain extent compared with the traditional method and get more reliable parameter estimations and its precision information,which validates the feasibility and effectiveness of the proposed method.

Recursive weighted least squares estimation algorithm based on minimum model error principle

Kalman filter is commonly used in data filtering and parameters estimation of nonlinear system,such as projectile's trajectory estimation and control.While there is a drawback that the prior error covariance matrix and filter parameters are difficult to be determined,which may result in filtering divergence.As to the problem that the accuracy of state estimation for nonlinear ballistic model strongly depends on its mathematical model,we improve the weighted least squares method(WLSM)with minimum model error principle.Invariant embedding method is adopted to solve the cost function including the model error.With the knowledge of measurement data and measurement error covariance matrix,we use gradient descent algorithm to determine the weighting matrix of model error.The uncertainty and linearization error of model are recursively estimated by the proposed method,thus achieving an online filtering estimation of the observations.Simulation results indicate that the proposed recursive estimation algorithm is insensitive to initial conditions and of good robustness.

Estimation under a Finite Mixture of Exponentiated Exponential Components Model and Balanced Square Error Loss

By exponentiating each of the components of a finite mixture of two exponential components model by a positive parameter, several shapes of hazard rate functions are obtained. Maximum likelihood and Bayes methods, based on square error loss function and objective prior, are used to obtain estimators based on balanced square error loss function for the parameters, survival and hazard rate functions of a mixture of two exponentiated exponential components model. Approximate interval estimators of the parameters of the model are obtained.

PENALIZED LEAST SQUARE IN SPARSE SETTING WITH CONVEX PENALTY AND NON GAUSSIAN ERRORS

This paper consider the penalized least squares estimators with convex penalties or regularization norms.We provide sparsity oracle inequalities for the prediction error for a general convex penalty and for the particular cases of Lasso and Group Lasso estimators in a regression setting.The main contribution is that our oracle inequalities are established for the more general case where the observations noise is issued from probability measures that satisfy a weak spectral gap(or Poincaré)inequality instead of Gaussian distributions.We illustrate our results on a heavy tailed example and a sub Gaussian one;we especially give the explicit bounds of the oracle inequalities for these two special examples.

Low Complexity Minimum Mean Square Error Channel Estimation for Adaptive Coding and Modulation Systems

Performance of the Adaptive Coding and Modulation(ACM) strongly depends on the retrieved Channel State Information(CSI),which can be obtained using the channel estimation techniques relying on pilot symbol transmission.Earlier analysis of methods of pilot-aided channel estimation for ACM systems were relatively little.In this paper,we investigate the performance of CSI prediction using the Minimum Mean Square Error(MMSE)channel estimator for an ACM system.To solve the two problems of MMSE:high computational operations and oversimplified assumption,we then propose the Low-Complexity schemes(LC-MMSE and Recursion LC-MMSE(R-LC-MMSE)).Computational complexity and Mean Square Error(MSE) are presented to evaluate the efficiency of the proposed algorithm.Both analysis and numerical results show that LC-MMSE performs close to the wellknown MMSE estimator with much lower complexity and R-LC-MMSE improves the application of MMSE estimation to specific circumstances.

Adaptive control of machining process based on extended entropy square error and wavelet neural network

Combining information entropy and wavelet analysis with neural network,an adaptive control system and an adaptive control algorithm are presented for machining process based on extended entropy square error(EESE)and wavelet neural network(WNN).Extended entropy square error function is defined and its availability is proved theoretically.Replacing the mean square error criterion of BP algorithm with the EESE criterion,the proposed system is then applied to the on-line control of the cutting force with variable cutting parameters by searching adaptively wavelet base function and self adjusting scaling parameter,translating parameter of the wavelet and neural network weights.Simulation results show that the designed system is of fast response,non-overshoot and it is more effective than the conventional adaptive control of machining process based on the neural network.The suggested algorithm can adaptively adjust the feed rate on-line till achieving a constant cutting force approaching the reference force in varied cutting conditions,thus improving the machining efficiency and protecting the tool.

ON THE SINGULARITY OF LEAST SQUARES ESTIMATOR FOR MEAN-REVERTING α-STABLE MOTIONS

We study the problem of parameter estimation for mean-reverting α-stable motion, dXt = （a0 - θ0Xt）dt ＋ dZt, observed at discrete time instants. A least squares estimator is obtained and its asymptotics is discussed in the singular case （a0, θ0） = （0, 0）. If a0 = 0, then the mean-reverting α-stable motion becomes Ornstein-Uhlenbeck process and is studied in [7] in the ergodic case θ0 〉 0. For the Ornstein-Uhlenbeck process, asymptotics of the least squares estimators for the singular case （θ0 = 0） and for ergodic case （θ0 〉 0） are completely different.

Improved adaptive pruning algorithm for least squares support vector regression

As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satis- factory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which in- volves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generali- zation performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.

Least Squares Solution for Discrete Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences

In this paper, an efficient computational approach is proposed to solve the discrete time nonlinear stochastic optimal control problem. For this purpose, a linear quadratic regulator model, which is a linear dynamical system with the quadratic criterion cost function, is employed. In our approach, the model-based optimal control problem is reformulated into the input-output equations. In this way, the Hankel matrix and the observability matrix are constructed. Further, the sum squares of output error is defined. In these point of views, the least squares optimization problem is introduced, so as the differences between the real output and the model output could be calculated. Applying the first-order derivative to the sum squares of output error, the necessary condition is then derived. After some algebraic manipulations, the optimal control law is produced. By substituting this control policy into the input-output equations, the model output is updated iteratively. For illustration, an example of the direct current and alternating current converter problem is studied. As a result, the model output trajectory of the least squares solution is close to the real output with the smallest sum squares of output error. In conclusion, the efficiency and the accuracy of the approach proposed are highly presented.

CHANNEL ERROR CORRECTION FOR WIDEBAND SAR-A JOINT MULTIPLE SUBPULSES PROCESSING METHOD

A novel method,referred to as joint multiple subpulses processing,is developed to calibrate the nonideal transfer function of radio frequency front-end and I/Q imbalance in quadrature modulate/demodulate systems simultaneously,which frequently occur in wideband Synthetic Aperture Radar(SAR) systems.Based on the time-frequency relation of the chirp signal and the analyses of the channel errors in wideband SAR,joint multiple subpulses processing method is adopted to separate the image frequency component due to the I/Q channel error.Then,the complete description of the channel error is acquired for building the correction function,which is used to correct the radar raw echo in frequency domain.The validity and capability of this method are demonstrated by the experiments of the channel error correction on the high resolution SAR system with the effective bandwidth of 500 MHz.

Stability of the MGS-like elimination method for equality constrained least squares problems

This paper proves that the weighting method via modified Gram-Schmidt（MGS） for solving the equality constrained least squares problem in the limit is equivalent to the direct elimination method via MGS（MGS-elimination method）. By virtue of this equivalence, the backward and forward roundoff error analysis of the MGS-elimination method is proved. Numerical experiments are provided to verify the results.

Tuning of Prior Covariance in Generalized Least Squares

Generalized Least Squares (least squares with prior information) requires the correct assignment of two prior covariance matrices: one associated with the uncertainty of measurements;the other with the uncertainty of prior information. These assignments often are very subjective, especially when correlations among data or among prior information are believed to occur. However, in cases in which the general form of these matrices can be anticipated up to a set of poorly-known parameters, the data and prior information may be used to better-determine (or “tune”) the parameters in a manner that is faithful to the underlying Bayesian foundation of GLS. We identify an objective function, the minimization of which leads to the best-estimate of the parameters and provide explicit and computationally-efficient formula for calculating the derivatives needed to implement the minimization with a gradient descent method. Furthermore, the problem is organized so that the minimization need be performed only over the space of covariance parameters, and not over the combined space of model and covariance parameters. We show that the use of trade-off curves to select the relative weight given to observations and prior information is not a form of tuning, because it does not, in general maximize the posterior probability of the model parameters, and can lead to a different weighting than the procedure described here. We also provide several examples that demonstrate the viability, and discuss both the advantages and limitations of the method.

A Squared-Chebyshev wavelet thresholding based 1D signal compression

In this paper a square wavelet thresholding method is proposed and evaluated as compared to the other classical wavelet thresholding methods (like soft and hard). The main advantage of this work is to design and implement a new wavelet thresholding method and evaluate it against other classical wavelet thresholding methods and hence search for the optimal wavelet mother function among the wide families with a suitable level of decomposition and followed by a novel thresholding method among the existing methods. This optimized method will be used to shrink the wavelet coefficients and yield an adequate compressed pressure signal prior to transmit it. While a comparison evaluation analysis is established, A new proposed procedure is used to compress a synthetic signal and obtain the optimal results through minimization the signal memory size and its transmission bandwidth. There are different performance indices to establish the comparison and evaluation process for signal compression;but the most well-known measuring scores are: NMSE, ESNR, and PDR. The obtained results showed the dominant of the square wavelet thresholding method against other methods using different measuring scores and hence the conclusion by the way for adopting this proposed novel wavelet thresholding method for 1D signal compression in future researches.

Improved cat swarm optimization for parameter estimation of mixed additive and multiplicative random error model

To estimate the parameters of the mixed additive and multiplicative(MAM)random error model using the weighted least squares iterative algorithm that requires derivation of the complex weight array,we introduce a derivative-free cat swarm optimization for parameter estimation.We embed the Powell method,which uses conjugate direction acceleration and does not need to derive the objective function,into the original cat swarm optimization to accelerate its convergence speed and search accuracy.We use the ordinary least squares,weighted least squares,original cat swarm optimization,particle swarm algorithm and improved cat swarm optimization to estimate the parameters of the straight-line fitting MAM model with lower nonlinearity and the DEM MAM model with higher nonlinearity,respectively.The experimental results show that the improved cat swarm optimization has faster convergence speed,higher search accuracy,and better stability than the original cat swarm optimization and the particle swarm algorithm.At the same time,the improved cat swarm optimization can obtain results consistent with the weighted least squares method based on the objective function only while avoiding multiple complex weight array derivations.The method in this paper provides a new idea for theoretical research on parameter estimation of MAM error models.

Galerkin-Petrov least squares mixed element method for stationary incompressible magnetohydrodynamics

The Galerkin-Petrov least squares method is combined with the mixed finite element method to deal with the stationary, incompressible magnetohydrodynamics system of equations with viscosity. A Galerkin-Petrov least squares mixed finite element format for the stationary incompressible magnetohydrodynamics equations is presented. And the existence and error estimates of its solution are derived. Through this method, the combination among the mixed finite element spaces does not demand the discrete Babuska-Brezzi stability conditions so that the mixed finite element spaces could be chosen arbitrartily and the error estimates with optimal order could be obtained.

A NONLINEAR GALERKIN/PETROV-LEAST SQUARES MIXED ELEMENT METHOD FOR THE STATIONARY NAVIER-STOKES EQUATIONS

A nonlinear Galerkin/Petrov-least squares mixed element (NGPLSME) method for the stationary Navier-Stokes equations is presented and analyzed. The scheme is that Petrov-least squares forms of residuals are added to the nonlinear Galerkin mixed element method so that it is stable for any combination of discrete velocity and pressure spaces without requiring the Babu*lka-Brezzi stability condition. The existence, uniqueness and convergence (at optimal rate) of the NGPLSME solution is proved in the case of sufficient viscosity (or small data).

Adaptive mixed least squares Galerkin/Petrov finite element method for stationary conduction convection problems

An adaptive mixed least squares Galerkin/Petrov finite element method （FEM） is developed for stationary conduction convection problems. The mixed least squares Galerkin/Petrov FEM is consistent and stable for any combination of discrete velocity and pressure spaces without requiring the Babuska-Brezzi stability condition. Using the general theory of Verfiirth, the posteriori error estimates of the residual type are derived. Finally, numerical tests are presented to illustrate the effectiveness of the method.

Condition for Successful Square Transformation in Time Series Modeling

In this study we establish the probability density function of the square transformed left-truncated N(1,σ2) error component of the multiplicative time series model and the functional expressions for its mean and variance. Furthermore the mean and variance of the square transformed left-truncated N(1,σ2) error component and those of the untransformed component were compared for the purpose of establishing the interval for σ where the properties of the two distributions are approximately the same in terms of equality of means and normality. From the results of the study, it was established that the two distributions are normally distributed and have means ≌1.0 correct to 1 dp in the interval 0 σ , hence a successful square transformation where necessary is achieved for values of σ such that 0 σ .