The Equivalence between Orthogonal Iterations and Alternating Least Squares

This note explores the relations between two different methods. The first one is the Alternating Least Squares (ALS) method for calculating a rank-k approximation of a real m×n matrix, A. This method has important applications in nonnegative matrix factorizations, in matrix completion problems, and in tensor approximations. The second method is called Orthogonal Iterations. Other names of this method are Subspace Iterations, Simultaneous Iterations, and block-Power method. Given a real symmetric matrix, G, this method computes k dominant eigenvectors of G. To see the relation between these methods we assume that G = AT A. It is shown that in this case the two methods generate the same sequence of subspaces, and the same sequence of low-rank approximations. This equivalence provides new insight into the convergence properties of both methods.

Two-Stage Procrustes Rotation with Sparse Target Matrix and Least Squares Criterion with Regularization and Generalized Weighting

In factor analysis, a factor loading matrix is often rotated to a simple target matrix for its simplicity. For the purpose, Procrustes rotation minimizes the discrepancy between the target and rotated loadings using two types of approximation: 1) approximate the zeros in the target by the non-zeros in the loadings, and 2) approximate the non-zeros in the target by the non-zeros in the loadings. The central issue of Procrustes rotation considered in the article is that it equally treats the two types of approximation, while the former is more important for simplifying the loading matrix. Furthermore, a well-known issue of Simplimax is the computational inefficiency in estimating the sparse target matrix, which yields a considerable number of local minima. The research proposes a new rotation procedure that consists of the following two stages. The first stage estimates sparse target matrix with lesser computational cost by regularization technique. In the second stage, a loading matrix is rotated to the target, emphasizing on the approximation of non-zeros to zeros in the target by least squares criterion with generalized weighing that is newly proposed by the study. The simulation study and real data examples revealed that the proposed method surely simplifies loading matrices.

Fast PARAFAC decomposition with application to polarization sensitive array parameter estimations

In tensor theory, the parallel factorization （PARAFAC）decomposition expresses a tensor as the sum of a set of rank-1tensors. By carrying out this numerical decomposition, mixedsources can be separated or unknown system parameters can beidentified, which is the so-called blind source separation or blindidentification. In this paper we propose a numerical PARAFACdecomposition algorithm. Compared to traditional algorithms, wespeed up the decomposition in several aspects, i.e., search di-rection by extrapolation, suboptimal step size by Gauss-Newtonapproximation, and linear search by n steps. The algorithm is ap-plied to polarization sensitive array parameter estimation to showits usefulness. Simulations verify the correctness and performanceof the proposed numerical techniques.

Randomized Algorithms for Orthogonal Nonnegative Matrix Factorization

Orthogonal nonnegative matrix factorization(ONMF)is widely used in blind image separation problem,document classification,and human face recognition.The model of ONMF can be efficiently solved by the alternating direction method of multipliers and hierarchical alternating least squares method.When the given matrix is huge,the cost of computation and communication is too high.Therefore,ONMF becomes challenging in the large-scale setting.The random projection is an efficient method of dimensionality reduction.In this paper,we apply the random projection to ONMF and propose two randomized algorithms.Numerical experiments show that our proposed algorithms perform well on both simulated and real data.

EFFICIENT NONNEGATIVE MATRIX FACTORIZATION VIA MODIFIED MONOTONE BARZILAI-BORWEIN METHOD WITH ADAPTIVE STEP SIZES STRATEGY

In this paper,we develop an active set identification technique.By means of the active set technique,we present an active set adaptive monotone projected Barzilai-Borwein method(ASAMPBB)for solving nonnegative matrix factorization(NMF)based on the alternating nonnegative least squares framework,in which the Barzilai-Borwein(BB)step sizes can be adaptively picked to get meaningful convergence rate improvements.To get optimal step size,we take into account of the curvature information.In addition,the larger step size technique is exploited to accelerate convergence of the proposed method.The global convergence of the proposed method is analysed under mild assumption.Finally,the results of the numerical experiments on both synthetic and real-world datasets show that the proposed method is effective.

Identification and application of nonlinear dynamic load models

An alternating least squares approach is developed in this paper to identify the exponential recovery dy- namic load model of wide-area power systems. The nonlinear optimization problem is decomposed to two linear least squares problems, and solved in an alternating way. Then, a new algorithm for numerical derivative calculation using dis- crete Fourier transform is proposed to attenuate the effect of noises in the process of parameter estimation. Based on the estimated dynamic load characteristics, the application on voltage stability is analyzed. Finally, numerical and laboratory examples are conducted to demonstrate the effectiveness of the orooosed methods.