Continuous Iteratively Reweighted Least Squares Algorithm for Solving Linear Models by Convex Relaxation

In this paper, we present continuous iteratively reweighted least squares algorithm (CIRLS) for solving the linear models problem by convex relaxation, and prove the convergence of this algorithm. Under some conditions, we give an error bound for the algorithm. In addition, the numerical result shows the efficiency of the algorithm.

An iterative algorithm for solving ill-conditioned linear least squares problems

Linear Least Squares（LLS） problems are particularly difficult to solve because they are frequently ill-conditioned, and involve large quantities of data. Ill-conditioned LLS problems are commonly seen in mathematics and geosciences, where regularization algorithms are employed to seek optimal solutions. For many problems, even with the use of regularization algorithms it may be impossible to obtain an accurate solution. Riley and Golub suggested an iterative scheme for solving LLS problems. For the early iteration algorithm, it is difficult to improve the well-conditioned perturbed matrix and accelerate the convergence at the same time. Aiming at this problem, self-adaptive iteration algorithm（SAIA） is proposed in this paper for solving severe ill-conditioned LLS problems. The algorithm is different from other popular algorithms proposed in recent references. It avoids matrix inverse by using Cholesky decomposition, and tunes the perturbation parameter according to the rate of residual error decline in the iterative process. Example shows that the algorithm can greatly reduce iteration times, accelerate the convergence,and also greatly enhance the computation accuracy.

The Equivalence between Orthogonal Iterations and Alternating Least Squares

This note explores the relations between two different methods. The first one is the Alternating Least Squares (ALS) method for calculating a rank-k approximation of a real m×n matrix, A. This method has important applications in nonnegative matrix factorizations, in matrix completion problems, and in tensor approximations. The second method is called Orthogonal Iterations. Other names of this method are Subspace Iterations, Simultaneous Iterations, and block-Power method. Given a real symmetric matrix, G, this method computes k dominant eigenvectors of G. To see the relation between these methods we assume that G = AT A. It is shown that in this case the two methods generate the same sequence of subspaces, and the same sequence of low-rank approximations. This equivalence provides new insight into the convergence properties of both methods.

Separating iterative solution model of generalized nonlinear dynamic least squares for data processing in building of digital earth

Data coming from different sources have different types and temporal states. Relations between one type of data and another ones, or between data and unknown parameters are almost nonlinear. It is not accurate and reliable to process the data in building the digital earth with the classical least squares method or the method of the common nonlinear least squares. So a generalized nonlinear dynamic least squares method was put forward to process data in building the digital earth. A separating solution model and the iterative calculation method were used to solve the generalized nonlinear dynamic least squares problem. In fact, a complex problem can be separated and then solved by converting to two sub problems, each of which has a single variable. Therefore the dimension of unknown parameters can be reduced to its half, which simplifies the original high dimensional equations.

Distributed Least-Squares Iterative Methods in Large-Scale Networks:A Survey

Many science and engineering applications involve solvinga linear least-squares system formed from some field measurements. In the distributed cyber-physical systems(CPS),each sensor node used for measurement often only knowspartial independent rows of the least-squares system. To solve the least-squares all the measurements must be gathered at a centralized location and then perform the computa-tion. Such data collection and computation are inefficient because of bandwidth and time constraints and sometimes areinfeasible because of data privacy concerns. Iterative methods are natural candidates for solving the aforementionedproblem and there are many studies regarding this. However,most of the proposed solutions are related to centralized/parallel computations while only a few have the potential to beapplied in distributed networks. Thus distributed computations are strongly preferred or demanded in many of the realworld applications, e.g. smart-grid, target tracking, etc. Thispaper surveys the representative iterative methods for distributed least-squares in networks.

A Method for Assessing Customer Harmonic Emission Level Based on the Iterative Algorithm for Least Square Estimation

With the power system harmonic pollution problems becoming more and more serious, how to distinguish the harmonic responsibility accurately and solve the grid harmonics simply and effectively has become the main development direction in harmonic control subjects. This paper, based on linear regression analysis of basic equation and improvement equation, deduced the least squares estimation (LSE) iterative algorithm and obtained the real-time estimates of regression coefficients, and then calculated the level of the harmonic impedance and emission estimates in real time. This paper used power system simulation software Matlab/Simulink as analysis tool and analyzed the user side of the harmonic amplitude and phase fluctuations PCC (point of common coupling) at the harmonic emission level, thus the research has a certain theoretical significance. The development of this algorithm combined with the instrument can be used in practical engineering.

Total least-squares EIO model,algorithms and applications

A functional model named EIO(Errors-In-Observations) is proposed for general TLS(total least-squares)adjustment. The EIO model only considers the correction of the observation vector, but doesn't consider to correct all elements in the design matrix as the EIV(Errors-In-Variables) model does, furthermore, the dimension of cofactor matrix is much smaller. Iterative algorithms for the parameter estimation and their precise covariance matrix are derived rigorously, and the computation steps are also presented. The proposed approach considers the correction of the observations in the coefficient matrix, and ensures their agreements in every matrix elements. Parameters and corrections can be solved at the same time.An approximate solution and a precise solution of the covariance matrix can be achieved by corresponding algorithms. Applications of EIO model and the proposed algorithms are demonstrated with several examples. The results and comparative studies show that the proposed EIO model and algorithms are feasible and reliable for general adjustment problems.

ON THE BREAKDOWNS OF THE GALERKIN AND LEAST-SQUARES METHODS

The Galerkin and least-squares methods are two classes of the most popular Krylov subspace methOds for solving large linear systems of equations. Unfortunately, both the methods may suffer from serious breakdowns of the same type: In a breakdown situation the Galerkin method is unable to calculate an approximate solution, while the least-squares method, although does not really break down, is unsucessful in reducing the norm of its residual. In this paper we first establish a unified theorem which gives a relationship between breakdowns in the two methods. We further illustrate theoretically and experimentally that if the coefficient matrix of a lienar system is of high defectiveness with the associated eigenvalues less than 1, then the restarted Galerkin and least-squares methods will be in great risks of complete breakdowns. It appears that our findings may help to understand phenomena observed practically and to derive treatments for breakdowns of this type.

Improved scheme to accelerate sparse least squares support vector regression

The pruning algorithms for sparse least squares support vector regression machine are common methods, and easily com- prehensible, but the computational burden in the training phase is heavy due to the retraining in performing the pruning process, which is not favorable for their applications. To this end, an im- proved scheme is proposed to accelerate sparse least squares support vector regression machine. A major advantage of this new scheme is based on the iterative methodology, which uses the previous training results instead of retraining, and its feasibility is strictly verified theoretically. Finally, experiments on bench- mark data sets corroborate a significant saving of the training time with the same number of support vectors and predictive accuracy compared with the original pruning algorithms, and this speedup scheme is also extended to classification problem.

Solving method of generalized nonlinear dynamic least squares for data processing in building of digital mine

Data are very important to build the digital mine. Data come from many sources, have different types and temporal states. Relations between one class of data and the other one, or between data and unknown parameters are more nonlinear. The unknown parameters are non random or random, among which the random parameters often dynamically vary with time. Therefore it is not accurate and reliable to process the data in building the digital mine with the classical least squares method or the method of the common nonlinear least squares. So a generalized nonlinear dynamic least squares method to process data in building the digital mine is put forward. In the meantime, the corresponding mathematical model is also given. The generalized nonlinear least squares problem is more complex than the common nonlinear least squares problem and its solution is more difficultly obtained because the dimensions of data and parameters in the former are bigger. So a new solution model and the method are put forward to solve the generalized nonlinear dynamic least squares problem. In fact, the problem can be converted to two sub problems, each of which has a single variable. That is to say, a complex problem can be separated and then solved. So the dimension of unknown parameters can be reduced to its half, which simplifies the original high dimensional equations. The method lessens the calculating load and opens up a new way to process the data in building the digital mine, which have more sources, different types and more temporal states.

Vibration Suppression for Active Magnetic Bearings Using Adaptive Filter with Iterative Search Algorithm

Active Magnetic Bearing(AMB) is a kind of electromagnetic support that makes the rotor movement frictionless and can suppress rotor vibration by controlling the magnetic force. The most common approach to restrain the rotor vibration in AMBs is to adopt a notch filter or adaptive filter in the AMB controller. However, these methods cannot obtain the precise amplitude and phase of the compensation current. Thus, they are not so effective in terms of suppressing the vibrations of the fundamental and other harmonic orders over the whole speed range. To improve the vibration suppression performance of AMBs,an adaptive filter based on Least Mean Square(LMS) is applied to extract the vibration signals from the rotor displacement signal. An Iterative Search Algorithm(ISA) is proposed in this paper to obtain the corresponding relationship between the compensation current and vibration signals. The ISA is responsible for searching the compensating amplitude and shifting phase online for the LMS filter, enabling the AMB controller to generate the corresponding compensation force for vibration suppression. The results of ISA are recorded to suppress vibration using the Look-Up Table(LUT) in variable speed range. Comprehensive simulations and experimental validations are carried out in fixed and variable speed range, and the results demonstrate that by employing the ISA, vibrations of the fundamental and other harmonic orders are suppressed effectively.

Low-complexity signal detection for massive MIMO systems via trace iterative method

Linear minimum mean square error(MMSE)detection has been shown to achieve near-optimal performance for massive multiple-input multiple-output(MIMO)systems but inevitably involves complicated matrix inversion,which entails high complexity.To avoid the exact matrix inversion,a considerable number of implicit and explicit approximate matrix inversion based detection methods is proposed.By combining the advantages of both the explicit and the implicit matrix inversion,this paper introduces a new low-complexity signal detection algorithm.Firstly,the relationship between implicit and explicit techniques is analyzed.Then,an enhanced Newton iteration method is introduced to realize an approximate MMSE detection for massive MIMO uplink systems.The proposed improved Newton iteration significantly reduces the complexity of conventional Newton iteration.However,its complexity is still high for higher iterations.Thus,it is applied only for first two iterations.For subsequent iterations,we propose a novel trace iterative method(TIM)based low-complexity algorithm,which has significantly lower complexity than higher Newton iterations.Convergence guarantees of the proposed detector are also provided.Numerical simulations verify that the proposed detector exhibits significant performance enhancement over recently reported iterative detectors and achieves close-to-MMSE performance while retaining the low-complexity advantage for systems with hundreds of antennas.

Least-Squares Solutions of Generalized Sylvester Equation with Xi Satisfies Different Linear Constraint

In this paper, an iterative method is constructed to find the least-squares solutions of generalized Sylvester equation , where is real matrices group, and satisfies different linear constraint. By this iterative method, for any initial matrix group within a special constrained matrix set, a least squares solution group with satisfying different linear constraint can be obtained within finite iteration steps in the absence of round off errors, and the unique least norm least-squares solution can be obtained by choosing a special kind of initial matrix group. In addition, a minimization property of this iterative method is characterized. Finally, numerical experiments are reported to show the efficiency of the proposed method.

DQ变换和MUSIC算法在ITER磁体电源信号间谐波检测中的应用

随着国际热核聚变实验堆(International Thermonuclear Experimental Reactor,ITER)计划的逐步开展,保证ITER磁体电源系统的稳定运行显得尤为重要。文章采用将DQ变换和多信号分类(multiple signal classification,MUSIC)算法相结合的方法进行间谐波频率检测,信号的幅度和相位由最小二乘法来估计。DQ变换可以消除大幅度ITER基波分量,MUSIC算法可以通过矩阵特征分解检测出短数据条件下的谐波和间谐波,适用短时平稳的间谐波检测,两者相结合可以有效检测出大幅度基波附近存在小幅度间谐波。仿真实验表明,计算经DQ变换后检测出的ITER信号谐波频率时,取中间信号计算真实频谱较为正确,两侧信号则有较大的误差。

Low-Complexity Signal Detection Based on Relaxation Iteration Method in Massive MIMO Systems

Minimum mean square error(MMSE) detection algorithm can achieve nearly optimal performance when the number of antennas at the base station(BS) is large enough compared to the number of users. But the traditional MMSE involves complicated matrix inversion. In this paper, we propose a modified MMSE algorithm which exploits the channel characteristics occurring in massive multiple-input multipleoutput(MIMO) channels and the relaxation iteration(RI) method to avoid the matrix inversion. A proper initial solution is given to accelerate the convergence speed. In addition, we point out that the channel estimation scheme used in channel hardening-exploiting message passing(CHEMP) receiver is very appropriate for our proposed detection algorithm. Simulation results verify that the proposed algorithm can achieve very close performance of the traditional MMSE algorithm with a small number of iterations.

Multi-loop Constrained Iterative Model Predictive Control Using ARX -PLS Decoupling Structure

A multi-loop constrained model predictive control scheme based on autoregressive exogenous-partial least squares(ARX-PLS) framework is proposed to tackle the high dimension, coupled and constraints problems in industry processes due to safety limitation, environmental regulations, consumer specifications and physical restriction. ARX-PLS decoupling character enables to turn the multivariable model predictive control(MPC) controller design in original space into the multi-loop single input single output(SISO) MPC controllers design in latent space.An idea of iterative method is applied to decouple the constraints latent variables in PLS framework and recursive least square is introduced to identify ARX-PLS model. This algorithm is applied to a non-square simulation system and a stirred reactor for ethylene polymerizations comparing with adaptive internal model control(IMC) method based on ARX-PLS framework. Its application has shown that this method outperforms adaptive IMC method based on ARX-PLS framework to some extent.

An Iterative Learning Approach to Identify Fractional Order KiBaM Model

This paper discusses the parameter and differentiation order identification of continuous fractional order KiBaM models in ARX (autoregressive model with exogenous inputs) and OE (output error model) forms. The least squares method is applied to the identification of nonlinear and linear parameters, in which the Grünwald-Letnikov definition and short memory principle are applied to compute the fractional order derivatives. An adaptive P-type order learning law is proposed to estimate the differentiation order iteratively and accurately. Particularly, a unique estimation result and a fast convergence speed can be arrived by using the small gain strategy, which is unidirectional and has certain advantages than some state-of-art methods. The proposed strategy can be successfully applied to the nonlinear systems with quasi-linear characteristics. The numerical simulations are shown to validate the concepts. © 2017 Chinese Association of Automation.

A Low-Complexity Signal Detection Utilizing AOR Iterative Method for Massive MIMO Systems

Massive multiple-input multiple-output(MIMO) system is capable of substantially improving the spectral efficiency as well as the capacity of wireless networks relying on equipping a large number of antenna elements at the base stations. However, the excessively high computational complexity of the signal detection in massive MIMO systems imposes a significant challenge for practical hardware implementations. In this paper, we propose a novel minimum mean square error(MMSE) signal detection using the accelerated overrelaxation(AOR) iterative method without complicated matrix inversion, which is capable of reducing the overall complexity of the classical MMSE algorithm by an order of magnitude. Simulation results show that the proposed AOR-based method can approach the conventional MMSE signal detection with significant complexity reduction.

Particle filter based on iterated importance density function and parallel resampling

The design, analysis and parallel implementation of particle filter(PF) were investigated. Firstly, to tackle the particle degeneracy problem in the PF, an iterated importance density function(IIDF) was proposed, where a new term associating with the current measurement information(CMI) was introduced into the expression of the sampled particles. Through the repeated use of the least squares estimate, the CMI can be integrated into the sampling stage in an iterative manner, conducing to the greatly improved sampling quality. By running the IIDF, an iterated PF(IPF) can be obtained. Subsequently, a parallel resampling(PR) was proposed for the purpose of parallel implementation of IPF, whose main idea was the same as systematic resampling(SR) but performed differently. The PR directly used the integral part of the product of the particle weight and particle number as the number of times that a particle was replicated, and it simultaneously eliminated the particles with the smallest weights, which are the two key differences from the SR. The detailed implementation procedures on the graphics processing unit of IPF based on the PR were presented at last. The performance of the IPF, PR and their parallel implementations are illustrated via one-dimensional numerical simulation and practical application of passive radar target tracking.

Stochastic Iterative Learning Control With Faded Signals

Stochastic iterative learning control(ILC) is designed for solving the tracking problem of stochastic linear systems through fading channels. Consequently, the signals used in learning control algorithms are faded in the sense that a random variable is multiplied by the original signal. To achieve the tracking objective, a two-dimensional Kalman filtering method is used in this study to derive a learning gain matrix varying along both time and iteration axes. The learning gain matrix minimizes the trace of input error covariance. The asymptotic convergence of the generated input sequence to the desired input value is strictly proved in the mean-square sense. Both output and input fading are accounted for separately in turn, followed by a general formulation that both input and output fading coexists.Illustrative examples are provided to verify the effectiveness of the proposed schemes.