Algorithms and statistical analysis for linear structured weighted total least squares problem

Weighted total least squares(WTLS)have been regarded as the standard tool for the errors-in-variables(EIV)model in which all the elements in the observation vector and the coefficient matrix are contaminated with random errors.However,in many geodetic applications,some elements are error-free and some random observations appear repeatedly in different positions in the augmented coefficient matrix.It is called the linear structured EIV(LSEIV)model.Two kinds of methods are proposed for the LSEIV model from functional and stochastic modifications.On the one hand,the functional part of the LSEIV model is modified into the errors-in-observations(EIO)model.On the other hand,the stochastic model is modified by applying the Moore-Penrose inverse of the cofactor matrix.The algorithms are derived through the Lagrange multipliers method and linear approximation.The estimation principles and iterative formula of the parameters are proven to be consistent.The first-order approximate variance-covariance matrix(VCM)of the parameters is also derived.A numerical example is given to compare the performances of our proposed three algorithms with the STLS approach.Afterwards,the least squares(LS),total least squares(TLS)and linear structured weighted total least squares(LSWTLS)solutions are compared and the accuracy evaluation formula is proven to be feasible and effective.Finally,the LSWTLS is applied to the field of deformation analysis,which yields a better result than the traditional LS and TLS estimations.

Least Squares One-Class Support Tensor Machine

One-class classification problem has become a popular problem in many fields, with a wide range of applications in anomaly detection, fault diagnosis, and face recognition. We investigate the one-class classification problem for second-order tensor data. Traditional vector-based one-class classification methods such as one-class support vector machine (OCSVM) and least squares one-class support vector machine (LSOCSVM) have limitations when tensor is used as input data, so we propose a new tensor one-class classification method, LSOCSTM, which directly uses tensor as input data. On one hand, using tensor as input data not only enables to classify tensor data, but also for vector data, classifying it after high dimensionalizing it into tensor still improves the classification accuracy and overcomes the over-fitting problem. On the other hand, different from one-class support tensor machine (OCSTM), we use squared loss instead of the original loss function so that we solve a series of linear equations instead of quadratic programming problems. Therefore, we use the distance to the hyperplane as a metric for classification, and the proposed method is more accurate and faster compared to existing methods. The experimental results show the high efficiency of the proposed method compared with several state-of-the-art methods.

Revisiting Akaike’s Final Prediction Error and the Generalized Cross Validation Criteria in Regression from the Same Perspective: From Least Squares to Ridge Regression and Smoothing Splines

In regression, despite being both aimed at estimating the Mean Squared Prediction Error (MSPE), Akaike’s Final Prediction Error (FPE) and the Generalized Cross Validation (GCV) selection criteria are usually derived from two quite different perspectives. Here, settling on the most commonly accepted definition of the MSPE as the expectation of the squared prediction error loss, we provide theoretical expressions for it, valid for any linear model (LM) fitter, be it under random or non random designs. Specializing these MSPE expressions for each of them, we are able to derive closed formulas of the MSPE for some of the most popular LM fitters: Ordinary Least Squares (OLS), with or without a full column rank design matrix;Ordinary and Generalized Ridge regression, the latter embedding smoothing splines fitting. For each of these LM fitters, we then deduce a computable estimate of the MSPE which turns out to coincide with Akaike’s FPE. Using a slight variation, we similarly get a class of MSPE estimates coinciding with the classical GCV formula for those same LM fitters.

Two-Stage Procrustes Rotation with Sparse Target Matrix and Least Squares Criterion with Regularization and Generalized Weighting

In factor analysis, a factor loading matrix is often rotated to a simple target matrix for its simplicity. For the purpose, Procrustes rotation minimizes the discrepancy between the target and rotated loadings using two types of approximation: 1) approximate the zeros in the target by the non-zeros in the loadings, and 2) approximate the non-zeros in the target by the non-zeros in the loadings. The central issue of Procrustes rotation considered in the article is that it equally treats the two types of approximation, while the former is more important for simplifying the loading matrix. Furthermore, a well-known issue of Simplimax is the computational inefficiency in estimating the sparse target matrix, which yields a considerable number of local minima. The research proposes a new rotation procedure that consists of the following two stages. The first stage estimates sparse target matrix with lesser computational cost by regularization technique. In the second stage, a loading matrix is rotated to the target, emphasizing on the approximation of non-zeros to zeros in the target by least squares criterion with generalized weighing that is newly proposed by the study. The simulation study and real data examples revealed that the proposed method surely simplifies loading matrices.

Robust least squares projection twin SVM and its sparse solution

Least squares projection twin support vector machine(LSPTSVM)has faster computing speed than classical least squares support vector machine(LSSVM).However,LSPTSVM is sensitive to outliers and its solution lacks sparsity.Therefore,it is difficult for LSPTSVM to process large-scale datasets with outliers.In this paper,we propose a robust LSPTSVM model(called R-LSPTSVM)by applying truncated least squares loss function.The robustness of R-LSPTSVM is proved from a weighted perspective.Furthermore,we obtain the sparse solution of R-LSPTSVM by using the pivoting Cholesky factorization method in primal space.Finally,the sparse R-LSPTSVM algorithm(SR-LSPTSVM)is proposed.Experimental results show that SR-LSPTSVM is insensitive to outliers and can deal with large-scale datasets fastly.

Least Square Finite Element Model for Analysis of Multilayered Composite Plates under Arbitrary Boundary Conditions

Laminated composites are widely used in many engineering industries such as aircraft, spacecraft, boat hulls, racing car bodies, and storage tanks. We analyze the 3D deformations of a multilayered, linear elastic, anisotropic rectangular plate subjected to arbitrary boundary conditions on one edge and simply supported on other edge. The rectangular laminate consists of anisotropic and homogeneous laminae of arbitrary thicknesses. This study presents the elastic analysis of laminated composite plates subjected to sinusoidal mechanical loading under arbitrary boundary conditions. Least square finite element solutions for displacements and stresses are investigated using a mathematical model, called a state-space model, which allows us to simultaneously solve for these field variables in the composite structure’s domain and ensure that continuity conditions are satisfied at layer interfaces. The governing equations are derived from this model using a numerical technique called the least-squares finite element method (LSFEM). These LSFEMs seek to minimize the squares of the governing equations and the associated side conditions residuals over the computational domain. The model is comprised of layerwise variables such as displacements, out-of-plane stresses, and in- plane strains, treated as independent variables. Numerical results are presented to demonstrate the response of the laminated composite plates under various arbitrary boundary conditions using LSFEM and compared with the 3D elasticity solution available in the literature.

Least-Squares及Galerkin谱元方法求解环形区域内的泊松方程

为研究基于Least-Squares变分及Galerkin变分两种形式的谱元方法的求解特性,推导了极坐标系中采用两种变分方法求解环形区域内Poisson方程时对应的弱解形式,采用Chebyshev多项式构造插值基函数进行空间离散,得到两种谱元方法对应的代数方程组,由此分析了系数矩阵结构的特点。数值计算结果显示:Least-Squares谱元方法为实现方程的降阶而引入新的求解变量,使得代数方程组形式更为复杂,但边界条件的处理比Galerkin谱元方法更为简单;两种谱元方法均能求解极坐标系中的Poisson方程且能获得高精度的数值解,二者绝对误差分布基本一致;固定单元内的插值阶数时,增加单元数可减小数值误差,且表现出代数精度的特点,误差降低速度较慢,而固定单元数时,在一定范围内数值误差随插值阶数的增加而减小的速度更快,表现出谱精度的特点;单元内插值阶数较高时,代数方程组系数矩阵的条件数急剧增多,方程组呈现病态,数值误差增大,这一特点限制了单元内插值阶数的取值。研究内容对深入了解两种谱元方法在极坐标系中求解Poisson方程时的特点、进一步采用相关分裂算法求解实际流动问题具有参考价值。

PSR-SQUARES:基于程序空间约简器的SQL逆向合成系统

针对SQUARES程序空间增长过快,导致程序合成效率偏低的问题,在SQUARES的基础上,增加了以深度神经网络为核心的程序空间约简器,将给定的<被查询表,查询结果>示例表示成二维张量,作为深度神经网络的输入,网络的输出是关于目标SQL语句合成规则的相关性标记向量。约简器根据神经网络的输出结果,采用末N位淘汰策略,删除与目标SQL语句相关性弱的合成规则,以减少候选SQL语句的生成和验证,提升系统合成效率。对约简器中深度神经网络的结构设计、训练样本集的生成方法和网络训练过程进行了详细描述。同时将PSR-SQUARES与当前有代表性SQL逆向合成系统进行实验对比,实验结果表明,PSR-SQUARES的综合性能不同程度地优于其他合成系统,平均合成时间由SQUARES的251 s降低至130 s,目标程序合成成功率由80%提升至89%。

Least Squares Evaluations for Form and Profile Errors of Ellipse Using Coordinate Data

To improve the measurement and evaluation of form error of an elliptic section, an evaluation method based on least squares fitting is investigated to analyze the form and profile errors of an ellipse using coordinate data. Two error indicators for defining ellipticity are discussed, namely the form error and the profile error, and the difference between both is considered as the main parameter for evaluating machining quality of surface and profile. Because the form error and the profile error rely on different evaluation benchmarks, the major axis and the foci rather than the centre of an ellipse are used as the evaluation benchmarks and can accurately evaluate a tolerance range with the separated form error and profile error of workpiece. Additionally, an evaluation program based on the LS model is developed to extract the form error and the profile error of the elliptic section, which is well suited for separating the two errors by a standard program. Finally, the evaluation method about the form and profile errors of the ellipse is applied to the measurement of skirt line of the piston, and results indicate the effectiveness of the evaluation. This approach provides the new evaluation indicators for the measurement of form and profile errors of ellipse, which is found to have better accuracy and can thus be used to solve the difficult of the measurement and evaluation of the piston in industrial production.

FAST RECURSIVE LEAST SQUARES LEARNING ALGORITHM FOR PRINCIPAL COMPONENT ANALYSIS

Based on the least-square minimization a computationally efficient learning algorithm for the Principal Component Analysis(PCA) is derived. The dual learning rate parameters are adaptively introduced to make the proposed algorithm providing the capability of the fast convergence and high accuracy for extracting all the principal components. It is shown that all the information needed for PCA can be completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input-output product. The convergence performance of the proposed algorithm is briefly analyzed.The relation between Oja’s rule and the least squares learning rule is also established. Finally, a simulation example is given to illustrate the effectiveness of this algorithm for PCA.

A hybrid inversion method of damped least squares with simulated annealing used for Rayleigh wave dispersion curve inversion

Surface wave methods are becoming increasingly popular in many geotechnical applications and in earthquake seismology due to their noninvasive characteristics.Inverse surface wave dispersion curves are a crucial step in most surface wave methods.Many inversion methods have been applied to surface wave dispersion curve inversion,including linearized inversion and nonlinearized inversion methods.In this study,a hybrid inversion method of Damped Least Squares(DLS) with Very Fast Simulated Annealing(VFSA) is developed for multi-mode Rayleigh wave dispersion curve inversion.Both synthetic and in situ fi eld data were used to verify the validity of the proposed method.The results show that the proposed method is superior to the conventional VFSA method in aiming at global minimum,especially when parameter searching space is adjacent to real values of the parameters.The advantage of the new method is that it retains both the merits of VFSA for global search and DLS for local search.At high temperatures,the global search dominates the runs,while at a low temperatures,the local search dominates the runs.Thus,at low temperatures,the proposed method can almost directly approach the actual model.

Properties of the total least squares estimation

Through theoretical derivation, some properties of the total least squares estimation are found. The total least squares estimation is the linear transformation of the least squares estimation, and the total least squares estimation is unbiased. The condition number of the total least squares estimation is greater than the least squares estimation, so the total least squares estimation is easier to be affected by the data error than the least squares estimation. Then through the further derivation, the relationships of solutions, residuals and unit weight variance estimations between the total least squares and the least squares are given.

Estimating Wheat Grain Protein Content Using Multi-Temporal Remote Sensing Data Based on Partial Least Squares Regression

Estimating wheat grain protein content by remote sensing is important for assessing wheat quality at maturity and making grains harvest and purchase policies. However, spatial variability of soil condition, temperature, and precipitation will affect grain protein contents and these factors usually cannot be monitored accurately by remote sensing data from single image. In this research, the relationships between wheat protein content at maturity and wheat agronomic parameters at different growing stages were analyzed and multi-temporal images of Landsat TM were used to estimate grain protein content by partial least squares regression. Experiment data were acquired in the suburb of Beijing during a 2-yr experiment in the period from 2003 to 2004. Determination coefficient, average deviation of self-modeling, and deviation of cross- validation were employed to assess the estimation accuracy of wheat grain protein content. Their values were 0.88, 1.30%, 3.81% and 0.72, 5.22%, 12.36% for 2003 and 2004, respectively. The research laid an agronomic foundation for GPC （grain protein content） estimation by multi-temporal remote sensing. The results showed that it is feasible to estimate GPC of wheat from multi-temporal remote sensing data in large area.

On-Line Batch Process Monitoring Using Multiway Kernel Partial Least Squares

An approach for batch processes monitoring and fault detection based on multiway kernel partial least squares(MKPLS) was presented.It is known that conventional batch process monitoring methods,such as multiway partial least squares(MPLS),are not suitable due to their intrinsic linearity when the variations are nonlinear.To address this issue,kernel partial least squares(KPLS) was used to capture the nonlinear relationship between the latent structures and predictive variables.In addition,KPLS requires only linear algebra and does not involve any nonlinear optimization.In this paper,the application of KPLS was extended to on-line monitoring of batch processes.The proposed batch monitoring method was applied to a simulation benchmark of fed-batch penicillin fermentation process.And the results demonstrate the superior monitoring performance of MKPLS in comparison to MPLS monitoring.

A multivariate partial least squares approach to joint association analysis for multiple correlated traits

Many complex traits are highly correlated rather than independent. By taking the correlation structure of multiple traits into account, joint association analyses can achieve both higher statistical power and more accurate estimation. To develop a statistical approach to joint association analysis that includes allele detection and genetic effect estimation, we combined multivariate partial least squares regression with variable selection strategies and selected the optimal model using the Bayesian Information Criterion(BIC). We then performed extensive simulations under varying heritabilities and sample sizes to compare the performance achieved using our method with those obtained by single-trait multilocus methods. Joint association analysis has measurable advantages over single-trait methods, as it exhibits superior gene detection power, especially for pleiotropic genes. Sample size, heritability,polymorphic information content(PIC), and magnitude of gene effects influence the statistical power, accuracy and precision of effect estimation by the joint association analysis.

LEAST SQUARES ESTIMATION FOR ORNSTEIN-UHLENBECK PROCESSES DRIVEN BY THE WEIGHTED FRACTIONAL BROWNIAN MOTION

In this article, we study a least squares estimator （LSE） of θ for the Ornstein- Uhlenbeck process X0=0,dXt=θXtdt＋dBt^ab, t ≥ 0 driven by weighted fractional Brownian motion B^a,b with parameters a, b. We obtain the consistency and the asymptotic distribution of the LSE based on the observation {Xs, s∈[0,t]} as t tends to infinity.

Discrimination of Transgenic Rice Based on Near Infrared Reflectance Spectroscopy and Partial Least Squares Regression Discriminant Analysis

Near infrared reflectance spectroscopy （NIRS）, a non-destructive measurement technique, was combined with partial least squares regression discrimiant analysis （PLS-DA） to discriminate the transgenic （TCTP and mi166） and wild type （Zhonghua 11） rice. Furthermore, rice lines transformed with protein gene （OsTCTP） and regulation gene （Osmi166） were also discriminated by the NIRS method. The performances of PLS-DA in spectral ranges of 4 000-8 000 cm-1 and 4 000-10 000 cm-1 were compared to obtain the optimal spectral range. As a result, the transgenic and wild type rice were distinguished from each other in the range of 4 000-10 000 cm-1, and the correct classification rate was 100.0% in the validation test. The transgenic rice TCTP and mi166 were also distinguished from each other in the range of 4 000-10 000 cm-1, and the correct classification rate was also 100.0%. In conclusion, NIRS combined with PLS-DA can be used for the discrimination of transgenic rice.

Least Squares Solution for Discrete Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences

In this paper, an efficient computational approach is proposed to solve the discrete time nonlinear stochastic optimal control problem. For this purpose, a linear quadratic regulator model, which is a linear dynamical system with the quadratic criterion cost function, is employed. In our approach, the model-based optimal control problem is reformulated into the input-output equations. In this way, the Hankel matrix and the observability matrix are constructed. Further, the sum squares of output error is defined. In these point of views, the least squares optimization problem is introduced, so as the differences between the real output and the model output could be calculated. Applying the first-order derivative to the sum squares of output error, the necessary condition is then derived. After some algebraic manipulations, the optimal control law is produced. By substituting this control policy into the input-output equations, the model output is updated iteratively. For illustration, an example of the direct current and alternating current converter problem is studied. As a result, the model output trajectory of the least squares solution is close to the real output with the smallest sum squares of output error. In conclusion, the efficiency and the accuracy of the approach proposed are highly presented.

Constrained Weighted Least Squares Location Algorithm Using Received Signal Strength Measurements

Determine the location of a target has gained considerable interest over the past few years. The Received Signal Strength(RSS) measurements and Differential RSS(DRSS) measurements can be converted to distance or distance ratio estimates for constructing a set of linear equations. Based on these linear equations, a constrained weighted least Squares(CWLS) algorithm for target localization is derived. In addition, an iterative technique based on Newton's method is utilized to give a solution. The covariance and bias of the CWLS algorithm is derived using perturbation analysis. Simulation shows that the proposed estimator achieves better performance than existing algorithms with reasonable complexity.

Temperature prediction control based on least squares support vector machines

A prediction control algorithm is presented based on least squares support vector machines (LS-SVM) model for a class of complex systems with strong nonlinearity. The nonlinear off-line model of the controlled plant is built by LS-SVM with radial basis function (RBF) kernel. In the process of system running, the off-line model is linearized at each sampling instant, and the generalized prediction control (GPC) algorithm is employed to implement the prediction control for the controlled plant. The obtained algorithm is applied to a boiler temperature control system with complicated nonlinearity and large time delay. The results of the experiment verify the effectiveness and merit of the algorithm.