Algorithms and statistical analysis for linear structured weighted total least squares problem

Weighted total least squares(WTLS)have been regarded as the standard tool for the errors-in-variables(EIV)model in which all the elements in the observation vector and the coefficient matrix are contaminated with random errors.However,in many geodetic applications,some elements are error-free and some random observations appear repeatedly in different positions in the augmented coefficient matrix.It is called the linear structured EIV(LSEIV)model.Two kinds of methods are proposed for the LSEIV model from functional and stochastic modifications.On the one hand,the functional part of the LSEIV model is modified into the errors-in-observations(EIO)model.On the other hand,the stochastic model is modified by applying the Moore-Penrose inverse of the cofactor matrix.The algorithms are derived through the Lagrange multipliers method and linear approximation.The estimation principles and iterative formula of the parameters are proven to be consistent.The first-order approximate variance-covariance matrix(VCM)of the parameters is also derived.A numerical example is given to compare the performances of our proposed three algorithms with the STLS approach.Afterwards,the least squares(LS),total least squares(TLS)and linear structured weighted total least squares(LSWTLS)solutions are compared and the accuracy evaluation formula is proven to be feasible and effective.Finally,the LSWTLS is applied to the field of deformation analysis,which yields a better result than the traditional LS and TLS estimations.

Least Squares One-Class Support Tensor Machine

One-class classification problem has become a popular problem in many fields, with a wide range of applications in anomaly detection, fault diagnosis, and face recognition. We investigate the one-class classification problem for second-order tensor data. Traditional vector-based one-class classification methods such as one-class support vector machine (OCSVM) and least squares one-class support vector machine (LSOCSVM) have limitations when tensor is used as input data, so we propose a new tensor one-class classification method, LSOCSTM, which directly uses tensor as input data. On one hand, using tensor as input data not only enables to classify tensor data, but also for vector data, classifying it after high dimensionalizing it into tensor still improves the classification accuracy and overcomes the over-fitting problem. On the other hand, different from one-class support tensor machine (OCSTM), we use squared loss instead of the original loss function so that we solve a series of linear equations instead of quadratic programming problems. Therefore, we use the distance to the hyperplane as a metric for classification, and the proposed method is more accurate and faster compared to existing methods. The experimental results show the high efficiency of the proposed method compared with several state-of-the-art methods.

PSR-SQUARES:基于程序空间约简器的SQL逆向合成系统

针对SQUARES程序空间增长过快,导致程序合成效率偏低的问题,在SQUARES的基础上,增加了以深度神经网络为核心的程序空间约简器,将给定的<被查询表,查询结果>示例表示成二维张量,作为深度神经网络的输入,网络的输出是关于目标SQL语句合成规则的相关性标记向量。约简器根据神经网络的输出结果,采用末N位淘汰策略,删除与目标SQL语句相关性弱的合成规则,以减少候选SQL语句的生成和验证,提升系统合成效率。对约简器中深度神经网络的结构设计、训练样本集的生成方法和网络训练过程进行了详细描述。同时将PSR-SQUARES与当前有代表性SQL逆向合成系统进行实验对比,实验结果表明,PSR-SQUARES的综合性能不同程度地优于其他合成系统,平均合成时间由SQUARES的251 s降低至130 s,目标程序合成成功率由80%提升至89%。

Robust least squares projection twin SVM and its sparse solution

Least squares projection twin support vector machine(LSPTSVM)has faster computing speed than classical least squares support vector machine(LSSVM).However,LSPTSVM is sensitive to outliers and its solution lacks sparsity.Therefore,it is difficult for LSPTSVM to process large-scale datasets with outliers.In this paper,we propose a robust LSPTSVM model(called R-LSPTSVM)by applying truncated least squares loss function.The robustness of R-LSPTSVM is proved from a weighted perspective.Furthermore,we obtain the sparse solution of R-LSPTSVM by using the pivoting Cholesky factorization method in primal space.Finally,the sparse R-LSPTSVM algorithm(SR-LSPTSVM)is proposed.Experimental results show that SR-LSPTSVM is insensitive to outliers and can deal with large-scale datasets fastly.

Revisiting Akaike’s Final Prediction Error and the Generalized Cross Validation Criteria in Regression from the Same Perspective: From Least Squares to Ridge Regression and Smoothing Splines

In regression, despite being both aimed at estimating the Mean Squared Prediction Error (MSPE), Akaike’s Final Prediction Error (FPE) and the Generalized Cross Validation (GCV) selection criteria are usually derived from two quite different perspectives. Here, settling on the most commonly accepted definition of the MSPE as the expectation of the squared prediction error loss, we provide theoretical expressions for it, valid for any linear model (LM) fitter, be it under random or non random designs. Specializing these MSPE expressions for each of them, we are able to derive closed formulas of the MSPE for some of the most popular LM fitters: Ordinary Least Squares (OLS), with or without a full column rank design matrix;Ordinary and Generalized Ridge regression, the latter embedding smoothing splines fitting. For each of these LM fitters, we then deduce a computable estimate of the MSPE which turns out to coincide with Akaike’s FPE. Using a slight variation, we similarly get a class of MSPE estimates coinciding with the classical GCV formula for those same LM fitters.

Two-Stage Procrustes Rotation with Sparse Target Matrix and Least Squares Criterion with Regularization and Generalized Weighting

In factor analysis, a factor loading matrix is often rotated to a simple target matrix for its simplicity. For the purpose, Procrustes rotation minimizes the discrepancy between the target and rotated loadings using two types of approximation: 1) approximate the zeros in the target by the non-zeros in the loadings, and 2) approximate the non-zeros in the target by the non-zeros in the loadings. The central issue of Procrustes rotation considered in the article is that it equally treats the two types of approximation, while the former is more important for simplifying the loading matrix. Furthermore, a well-known issue of Simplimax is the computational inefficiency in estimating the sparse target matrix, which yields a considerable number of local minima. The research proposes a new rotation procedure that consists of the following two stages. The first stage estimates sparse target matrix with lesser computational cost by regularization technique. In the second stage, a loading matrix is rotated to the target, emphasizing on the approximation of non-zeros to zeros in the target by least squares criterion with generalized weighing that is newly proposed by the study. The simulation study and real data examples revealed that the proposed method surely simplifies loading matrices.

On-Line Batch Process Monitoring Using Multiway Kernel Partial Least Squares

An approach for batch processes monitoring and fault detection based on multiway kernel partial least squares(MKPLS) was presented.It is known that conventional batch process monitoring methods,such as multiway partial least squares(MPLS),are not suitable due to their intrinsic linearity when the variations are nonlinear.To address this issue,kernel partial least squares(KPLS) was used to capture the nonlinear relationship between the latent structures and predictive variables.In addition,KPLS requires only linear algebra and does not involve any nonlinear optimization.In this paper,the application of KPLS was extended to on-line monitoring of batch processes.The proposed batch monitoring method was applied to a simulation benchmark of fed-batch penicillin fermentation process.And the results demonstrate the superior monitoring performance of MKPLS in comparison to MPLS monitoring.

Perturbation Analysis for the Matrix-Scaled Total Least Squares Problem

In this paper, we extend matrix scaled total least squares (MSTLS) problem with a single right-hand side to the case of multiple right-hand sides. Firstly, under some mild conditions, this paper gives an explicit expression of the minimum norm solution of MSTLS problem with multiple right-hand sides. Then, we present the Kronecker-product-based formulae for the normwise, mixed and componentwise condition numbers of the MSTLS problem. For easy estimation, we also exhibit Kronecker-product-free upper bounds for these condition numbers. All these results can reduce to those of the total least squares (TLS) problem which were given by Zheng et al. Finally, two numerical experiments are performed to illustrate our results.

FAST RECURSIVE LEAST SQUARES LEARNING ALGORITHM FOR PRINCIPAL COMPONENT ANALYSIS

Based on the least-square minimization a computationally efficient learning algorithm for the Principal Component Analysis(PCA) is derived. The dual learning rate parameters are adaptively introduced to make the proposed algorithm providing the capability of the fast convergence and high accuracy for extracting all the principal components. It is shown that all the information needed for PCA can be completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input-output product. The convergence performance of the proposed algorithm is briefly analyzed.The relation between Oja’s rule and the least squares learning rule is also established. Finally, a simulation example is given to illustrate the effectiveness of this algorithm for PCA.

Least Squares Evaluations for Form and Profile Errors of Ellipse Using Coordinate Data

To improve the measurement and evaluation of form error of an elliptic section, an evaluation method based on least squares fitting is investigated to analyze the form and profile errors of an ellipse using coordinate data. Two error indicators for defining ellipticity are discussed, namely the form error and the profile error, and the difference between both is considered as the main parameter for evaluating machining quality of surface and profile. Because the form error and the profile error rely on different evaluation benchmarks, the major axis and the foci rather than the centre of an ellipse are used as the evaluation benchmarks and can accurately evaluate a tolerance range with the separated form error and profile error of workpiece. Additionally, an evaluation program based on the LS model is developed to extract the form error and the profile error of the elliptic section, which is well suited for separating the two errors by a standard program. Finally, the evaluation method about the form and profile errors of the ellipse is applied to the measurement of skirt line of the piston, and results indicate the effectiveness of the evaluation. This approach provides the new evaluation indicators for the measurement of form and profile errors of ellipse, which is found to have better accuracy and can thus be used to solve the difficult of the measurement and evaluation of the piston in industrial production.

A hybrid inversion method of damped least squares with simulated annealing used for Rayleigh wave dispersion curve inversion

Surface wave methods are becoming increasingly popular in many geotechnical applications and in earthquake seismology due to their noninvasive characteristics.Inverse surface wave dispersion curves are a crucial step in most surface wave methods.Many inversion methods have been applied to surface wave dispersion curve inversion,including linearized inversion and nonlinearized inversion methods.In this study,a hybrid inversion method of Damped Least Squares(DLS) with Very Fast Simulated Annealing(VFSA) is developed for multi-mode Rayleigh wave dispersion curve inversion.Both synthetic and in situ fi eld data were used to verify the validity of the proposed method.The results show that the proposed method is superior to the conventional VFSA method in aiming at global minimum,especially when parameter searching space is adjacent to real values of the parameters.The advantage of the new method is that it retains both the merits of VFSA for global search and DLS for local search.At high temperatures,the global search dominates the runs,while at a low temperatures,the local search dominates the runs.Thus,at low temperatures,the proposed method can almost directly approach the actual model.

Development of a least squares support vector machine model for prediction of natural gas hydrate formation temperature

Hydrates always are considered as a threat to petroleum industry due to the operational problems it can cause.These problems could result in reducing production performance or even production stoppage for a long time.In this paper, we were intended to develop a LSSVM algorithm for prognosticating hydrate formation temperature(HFT) in a wide range of natural gas mixtures. A total number of 279 experimental data points were extracted from open literature to develop the LSSVM. The input parameters were chosen based on the hydrate structure that each gas species form. The modeling resulted in a robust algorithm with the squared correlation coefficients(R^2) of 0.9918. Aside from the excellent statistical parameters of the model, comparing proposed LSSVM with some of conventional correlations showed its supremacy, particularly in the case of sour gases with high H_2S concentrations, where the model surpasses all correlations and existing thermodynamic models. For detection of the probable doubtful experimental data, and applicability of the model, the Leverage statistical approach was performed on the data sets. This algorithm showed that the proposed LSSVM model is statistically valid for HFT prediction and almost all the data points are in the applicability domain of the model.

A multivariate partial least squares approach to joint association analysis for multiple correlated traits

Many complex traits are highly correlated rather than independent. By taking the correlation structure of multiple traits into account, joint association analyses can achieve both higher statistical power and more accurate estimation. To develop a statistical approach to joint association analysis that includes allele detection and genetic effect estimation, we combined multivariate partial least squares regression with variable selection strategies and selected the optimal model using the Bayesian Information Criterion(BIC). We then performed extensive simulations under varying heritabilities and sample sizes to compare the performance achieved using our method with those obtained by single-trait multilocus methods. Joint association analysis has measurable advantages over single-trait methods, as it exhibits superior gene detection power, especially for pleiotropic genes. Sample size, heritability,polymorphic information content(PIC), and magnitude of gene effects influence the statistical power, accuracy and precision of effect estimation by the joint association analysis.

Local geoid height approximation and interpolation using moving least squares approach

In this paper an introduction of the moving least squares approach is presented in the context of data approximation and interpolation problems in Geodesy.An application of this method is presented for geoid height approximation and interpolation using different polynomial basis functions for the approximant and interpolant,respectively,in a regular grid of geoid height data in the region 16.0417°≤φ≤47.9583°and 36.0417°≤λ≤69.9582°,with increment 0.0833°in both latitudal and longitudal directions.The results of approximation and interpolation are then compared with the geoid height data from GPS-Levelling approach.Using the standard deviation of the difference of the results,it is shown that the planar interpolant,with reciprocal of distance as weight function,is the best choice in this local approximation and interpolation problem.

LEAST SQUARES ESTIMATION FOR ORNSTEIN-UHLENBECK PROCESSES DRIVEN BY THE WEIGHTED FRACTIONAL BROWNIAN MOTION

In this article, we study a least squares estimator(LSE) of θ for the OrnsteinUhlenbeck process X_0=0, dX_t =θX_tdt + dB_t^(a,b), t≥ 0 driven by weighted fractional Brownian motion B^(a,b) with parameters a, b. We obtain the consistency and the asymptotic distribution of the LSE based on the observation {X_s, s ∈ [0, t]} as t tends to infinity.

Constrained Weighted Least Squares Location Algorithm Using Received Signal Strength Measurements

Determine the location of a target has gained considerable interest over the past few years. The Received Signal Strength(RSS) measurements and Differential RSS(DRSS) measurements can be converted to distance or distance ratio estimates for constructing a set of linear equations. Based on these linear equations, a constrained weighted least Squares(CWLS) algorithm for target localization is derived. In addition, an iterative technique based on Newton's method is utilized to give a solution. The covariance and bias of the CWLS algorithm is derived using perturbation analysis. Simulation shows that the proposed estimator achieves better performance than existing algorithms with reasonable complexity.

Penalized total least squares method for dealing with systematic errors in partial EIV model and its precision estimation

When the total least squares(TLS)solution is used to solve the parameters in the errors-in-variables(EIV)model,the obtained parameter estimations will be unreliable in the observations containing systematic errors.To solve this problem,we propose to add the nonparametric part(systematic errors)to the partial EIV model,and build the partial EIV model to weaken the influence of systematic errors.Then,having rewritten the model as a nonlinear model,we derive the formula of parameter estimations based on the penalized total least squares criterion.Furthermore,based on the second-order approximation method of precision estimation,we derive the second-order bias and covariance of parameter estimations and calculate the mean square error(MSE).Aiming at the selection of the smoothing factor,we propose to use the U curve method.The experiments show that the proposed method can mitigate the influence of systematic errors to a certain extent compared with the traditional method and get more reliable parameter estimations and its precision information,which validates the feasibility and effectiveness of the proposed method.

Real-Time Patient-Specific ECG Arrhythmia Detection by Quantum Genetic Algorithm of Least Squares Twin SVM

The automatic detection of cardiac arrhythmias through remote monitoring is still a challenging task since electrocardiograms(ECGs)are easily contaminated by physiological artifacts and external noises,and these morphological characteristics show significant variations for different patients.A fast patient-specific arrhythmia diagnosis classifier scheme is proposed,in which a wavelet adaptive threshold denoising is combined with quantum genetic algorithm(QAG)based on least squares twin support vector machine(LSTSVM).The wavelet adaptive threshold denoising is employed for noise reduction,and then morphological features combined with the timing interval features are extracted to evaluate the classifier.For each patient,an individual and fast classifier will be trained by common and patient-specific training data.Following the recommendations of the Association for the Advancements of Medical Instrumentation(AAMI),experimental results over the MIT-BIH arrhythmia benchmark database demonstrated that our proposed method achieved the average detection accuracy of 98.22%,99.65%and 99.41%for the abnormal,ventricular ectopic beats(VEBs)and supra-VEBs(SVEBs),respectively.Besides the detection accuracy,sensitivity and specificity,our proposed method consumes the less CPU running time compared with the other representative state of the art methods.It can be ported to Android based embedded system,henceforth suitable for a wearable device.

Least Squares Solution for Discrete Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences

In this paper, an efficient computational approach is proposed to solve the discrete time nonlinear stochastic optimal control problem. For this purpose, a linear quadratic regulator model, which is a linear dynamical system with the quadratic criterion cost function, is employed. In our approach, the model-based optimal control problem is reformulated into the input-output equations. In this way, the Hankel matrix and the observability matrix are constructed. Further, the sum squares of output error is defined. In these point of views, the least squares optimization problem is introduced, so as the differences between the real output and the model output could be calculated. Applying the first-order derivative to the sum squares of output error, the necessary condition is then derived. After some algebraic manipulations, the optimal control law is produced. By substituting this control policy into the input-output equations, the model output is updated iteratively. For illustration, an example of the direct current and alternating current converter problem is studied. As a result, the model output trajectory of the least squares solution is close to the real output with the smallest sum squares of output error. In conclusion, the efficiency and the accuracy of the approach proposed are highly presented.

Recursive weighted least squares estimation algorithm based on minimum model error principle

Kalman filter is commonly used in data filtering and parameters estimation of nonlinear system,such as projectile's trajectory estimation and control.While there is a drawback that the prior error covariance matrix and filter parameters are difficult to be determined,which may result in filtering divergence.As to the problem that the accuracy of state estimation for nonlinear ballistic model strongly depends on its mathematical model,we improve the weighted least squares method(WLSM)with minimum model error principle.Invariant embedding method is adopted to solve the cost function including the model error.With the knowledge of measurement data and measurement error covariance matrix,we use gradient descent algorithm to determine the weighting matrix of model error.The uncertainty and linearization error of model are recursively estimated by the proposed method,thus achieving an online filtering estimation of the observations.Simulation results indicate that the proposed recursive estimation algorithm is insensitive to initial conditions and of good robustness.