PROJECTED GRADIENT DESCENT BASED ON SOFT THRESHOLDING IN MATRIX COMPLETION

Matrix completion is the extension of compressed sensing.In compressed sensing,we solve the underdetermined equations using sparsity prior of the unknown signals.However,in matrix completion,we solve the underdetermined equations based on sparsity prior in singular values set of the unknown matrix,which also calls low-rank prior of the unknown matrix.This paper firstly introduces basic concept of matrix completion,analyses the matrix suitably used in matrix completion,and shows that such matrix should satisfy two conditions:low rank and incoherence property.Then the paper provides three reconstruction algorithms commonly used in matrix completion:singular value thresholding algorithm,singular value projection,and atomic decomposition for minimum rank approximation,puts forward their shortcoming to know the rank of original matrix.The Projected Gradient Descent based on Soft Thresholding(STPGD),proposed in this paper predicts the rank of unknown matrix using soft thresholding,and iteratives based on projected gradient descent,thus it could estimate the rank of unknown matrix exactly with low computational complexity,this is verified by numerical experiments.We also analyze the convergence and computational complexity of the STPGD algorithm,point out this algorithm is guaranteed to converge,and analyse the number of iterations needed to reach reconstruction error.Compared the computational complexity of the STPGD algorithm to other algorithms,we draw the conclusion that the STPGD algorithm not only reduces the computational complexity,but also improves the precision of the reconstruction solution.

Projected gradient trust-region method for solving nonlinear systems with convex constraints

In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of computing trial directions by this method combining with the line search technique. Close to the solution set this method is locally Q-superlinearly convergent under an error bound assumption which is much weaker than the standard nonsingularity condition.

A projected gradient method with nonmonotonic backtracking technique for solving convex constrained monotone variational inequality problem

Based on a differentiable merit function proposed by Taji, et al in “Mathematical Programming, 1993, 58： 369-383”, a projected gradient trust region method for the monotone variational inequality problem with convex constraints is presented. Theoretical analysis is given which proves that the proposed algorithm is globally convergent and has a local quadratic convergence rate under some reasonable conditions. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

ANonmonotone Projected Gradient Method for Multiobjective Problems on Convex Sets

In this work we consider an extension of the classical scalar-valued projected gradient method for multiobjective problems on convex sets.As in Fazzio et al.(Optim Lett 13:1365-1379,2019)a parameter which controls the step length is considered and an updating rule based on the spectral gradient method from the scalar case is proposed.In the present paper,we consider an extension of the traditional nonmonotone approach of Grippo et al.(SIAM J Numer Anal 23:707-716,1986)based on the maximum of some previous function values as suggested in Mita et al.(J Glob Optim 75:539-559,2019)for unconstrained multiobjective optimization problems.We prove the accumulation points of sequences generated by the proposed algorithm,if they exist,are stationary points of the original problem.Numerical experiments are reported.

Adaptive projected gradient thresholding methods for constrained l0problems

In this paper, we propose and analyze adaptive projected gradient thresholding(APGT) methods for finding sparse solutions of the underdetermined linear systems with equality and box constraints. The general convergence will be demonstrated, and in addition, the bound of the number of iterations is established in some special cases. Under suitable assumptions, it is proved that any accumulation point of the sequence generated by the APGT methods is a local minimizer of the underdetermined linear system. Moreover, the APGT methods, under certain conditions, can find all s-sparse solutions for accurate measurement cases and guarantee the stability and robustness for flawed measurement cases. Numerical examples are presented to show the accordance with theoretical results in compressed sensing and verify high out-of-sample performance in index tracking.

Convergence analysis of projected gradient descent for Schatten-p nonconvex matrix recovery

The matrix rank minimization problem arises in many engineering applications. As this problem is NP-hard, a nonconvex relaxation of matrix rank minimization, called the Schatten-p quasi-norm minimization(0 < p < 1), has been developed to approximate the rank function closely. We study the performance of projected gradient descent algorithm for solving the Schatten-p quasi-norm minimization(0 < p < 1) problem.Based on the matrix restricted isometry property(M-RIP), we give the convergence guarantee and error bound for this algorithm and show that the algorithm is robust to noise with an exponential convergence rate.

A projected gradient based game theoretic approach for multi-user power control in cognitive radio network

The fifth generation （5G） networks have been envisioned to support the explosive growth of data demand caused by the increasing traditional high-rate mobile users and the expected rise of interconnections between human and things. To accommodate the ever-growing data traffic with scarce spectrum resources, cognitive radio （CR） is considered a promising technology to improve spectrum utilization. We study the power control problem for secondary users in an underlay CR network. Unlike most existing studies which simplify the problem by considering only a single primary user or channel, we investigate a more realistic scenario where multiple primary users share multiple channels with secondary users. We formulate the power control problem as a non-cooperative game with coupled constraints, where the Pareto optimality and achievable total throughput can be obtained by a Nash equilibrium （NE） solution. To achieve NE of the game, we first propose a projected gradient based dynamic model whose equilibrium points are equivalent to the NE of the original game, and then derive a centralized algorithm to solve the problem. Simulation results show that the convergence and effectiveness of our proposed solution, emphasizing the proposed algorithm, are competitive. Moreover, we demonstrate the robustness of our proposed solution as the network size increases.

A Framework of Convergence Analysis of Mini-batch Stochastic Projected Gradient Methods

In this paper,we establish a unified framework to study the almost sure global convergence and the expected convergencerates of a class ofmini-batch stochastic(projected)gradient(SG)methods,including two popular types of SG:stepsize diminished SG and batch size increased SG.We also show that the standard variance uniformly bounded assumption,which is frequently used in the literature to investigate the convergence of SG,is actually not required when the gradient of the objective function is Lipschitz continuous.Finally,we show that our framework can also be used for analyzing the convergence of a mini-batch stochastic extragradient method for stochastic variational inequality.

Discontinuous element pressure gradient stabilizations for compressible Navier-Stokes equations based on local projections

A pressure gradient discontinuous finite element formulation for the compressible Navier-Stokes equations is derived based on local projections. The resulting finite element formulation is stable and uniquely solvable without requiring a B-B stability condition. An error estimate is Obtained.

A New Conjugate Gradient Projection Method for Solving Stochastic Generalized Linear Complementarity Problems

In this paper, a class of the stochastic generalized linear complementarity problems with finitely many elements is proposed for the first time. Based on the Fischer-Burmeister function, a new conjugate gradient projection method is given for solving the stochastic generalized linear complementarity problems. The global convergence of the conjugate gradient projection method is proved and the related numerical results are also reported.

A Probe Method of Gradient Projection Type

In this paper,a probe method for nonlinear programming wiht equality and inequality is given. Its iterative directions at an arbitrary point x can be obtained through solving a liear system. The terminate conditions and choices of the parameters are given. The global convergence of the method is proved. Further more,some well known gradient projection type algorithms [1-15] and new gradient projection type algorithms from the linear system are given in this paper.

Reduced projection augmented Lagrange bi-conjugate gradient method for contact and impact problems

Based on the numerical governing formulation and non-linear complementary conditions of contact and impact problems, a reduced projection augmented Lagrange bi- conjugate gradient method is proposed for contact and impact problems by translating non-linear complementary conditions into equivalent formulation of non-linear program- ming. For contact-impact problems, a larger time-step can be adopted arriving at numer- ical convergence compared with penalty method. By establishment of the impact-contact formulations which are equivalent with original non-linear complementary conditions, a reduced projection augmented Lagrange bi-conjugate gradient method is deduced to im- prove precision and efficiency of numerical solutions. A numerical example shows that the algorithm we suggested is valid and exact.

Distributed Nash Equilibrium Seeking Strategies Under Quantized Communication

This paper is concerned with distributed Nash equi librium seeking strategies under quantized communication. In the proposed seeking strategy, a projection operator is synthesized with a gradient search method to achieve the optimization o players' objective functions while restricting their actions within required non-empty, convex and compact domains. In addition, a leader-following consensus protocol, in which quantized informa tion flows are utilized, is employed for information sharing among players. More specifically, logarithmic quantizers and uniform quantizers are investigated under both undirected and connected communication graphs and strongly connected digraphs, respec tively. Through Lyapunov stability analysis, it is shown that play ers' actions can be steered to a neighborhood of the Nash equilib rium with logarithmic and uniform quantizers, and the quanti fied convergence error depends on the parameter of the quan tizer for both undirected and directed cases. A numerical exam ple is given to verify the theoretical results.

Minimum distance constrained nonnegative matrix factorization for hyperspectral data unmixing

This paper considers a problem of unsupervised spectral unmixing of hyperspectral data. Based on the Linear Mixing Model （ LMM）, a new method under the framework of nonnegative matrix fac- torization （NMF） is proposed, namely minimum distance constrained nonnegative matrix factoriza- tion （MDC-NMF）. In this paper, firstly, a new regularization term, called endmember distance （ED） is considered, which is defined as the sum of the squared Euclidean distances from each end- member to their geometric center. Compared with the simplex volume, ED has better optimization properties and is conceptually intuitive. Secondly, a projected gradient （PG） scheme is adopted, and by the virtue of ED, in this scheme the optimal step size along the feasible descent direction can be calculated easily at each iteration. Thirdly, a finite step （ no more than the number of endmem- bers） terminated algorithm is used to project a point on the canonical simplex, by which the abun- dance nonnegative constraint and abundance sum-to-one constraint can be accurately satisfied in a light amount of computation. The experimental results, based on a set of synthetic data and real da- ta, demonstrate that, in the same running time, MDC-NMF outperforms several other similar meth- ods proposed recently.

Application of a Derivative-Free Method with Projection Skill to Solve an Optimization Problem

Improving numerical forecasting skill in the atmospheric and oceanic sciences by solving optimization problems is an important issue. One such method is to compute the conditional nonlinear optimal perturbation(CNOP), which has been applied widely in predictability studies. In this study, the Differential Evolution(DE) algorithm, which is a derivative-free algorithm and has been applied to obtain CNOPs for exploring the uncertainty of terrestrial ecosystem processes, was employed to obtain the CNOPs for finite-dimensional optimization problems with ball constraint conditions using Burgers' equation. The aim was first to test if the CNOP calculated by the DE algorithm is similar to that computed by traditional optimization algorithms, such as the Spectral Projected Gradient(SPG2) algorithm. The second motive was to supply a possible route through which the CNOP approach can be applied in predictability studies in the atmospheric and oceanic sciences without obtaining a model adjoint system, or for optimization problems with non-differentiable cost functions. A projection skill was first explanted to the DE algorithm to calculate the CNOPs. To validate the algorithm, the SPG2 algorithm was also applied to obtain the CNOPs for the same optimization problems. The results showed that the CNOPs obtained by the DE algorithm were nearly the same as those obtained by the SPG2 algorithm in terms of their spatial distributions and nonlinear evolutions. The implication is that the DE algorithm could be employed to calculate the optimal values of optimization problems, especially for non-differentiable and nonlinear optimization problems associated with the atmospheric and oceanic sciences.

NEW ALGORITHM FOR FIR FILTER DESIGN WITH DISCRETE COEFFICIENTS

This paper develops a new algorithm based on the Projected Gradient Algorithm (PGA) for the design of FIR digital filters with "sum of power of two" coefficients. It is shown that the integer programming involved in the FIR filter design can be solved by this algorithm. It is compared with the reported method for a SemiDefinite Programming (SDP) relaxation- based design. The simulations demonstrate that the new algorithm often yields the similar error performances of the FIR filter design, but the average CPU time of this approach is significantly reduced.

A Fast Algorithm for Training Large Scale Support Vector Machines

The manuscript presents an augmented Lagrangian—fast projected gradient method (ALFPGM) with an improved scheme of working set selection, pWSS, a decomposition based algorithm for training support vector classification machines (SVM). The manuscript describes the ALFPGM algorithm, provides numerical results for training SVM on large data sets, and compares the training times of ALFPGM and Sequential Minimal Minimization algorithms (SMO) from Scikit-learn library. The numerical results demonstrate that ALFPGM with the improved working selection scheme is capable of training SVM with tens of thousands of training examples in a fraction of the training time of some widely adopted SVM tools.

A FAST CONVERGING SPARSE RECONSTRUCTION ALGORITHM IN GHOST IMAGING

A fast converging sparse reconstruction algorithm in ghost imaging is presented. It utilizes total variation regularization and its formulation is based on the Karush-Kuhn-Tucker (KKT) theorem in the theory of convex optimization. Tests using experimental data show that, compared with the algorithm of Gradient Projection for Sparse Reconstruction (GPSR), the proposed algorithm yields better results with less computation work.

A multi-objective model for cordon-based congestion pricing schemes with nonlinear distance tolls

Congestion pricing is an important component of urban intelligent transport system.The efficiency,equity and the environmental impacts associated with road pricing schemes are key issues that should be considered before such schemes are implemented.This paper focuses on the cordon-based pricing with distance tolls,where the tolls are determined by a nonlinear function of a vehicles' travel distance within a cordon,termed as toll charge function.The optimal tolls can give rise to:1) higher total social benefits,2) better levels of equity,and 3) reduced environmental impacts(e.g.,less emission).Firstly,a deterministic equilibrium(DUE) model with elastic demand is presented to evaluate any given toll charge function.The distance tolls are non-additive,thus a modified path-based gradient projection algorithm is developed to solve the DUE model.Then,to quantitatively measure the equity level of each toll charge function,the Gini coefficient is adopted to measure the equity level of the flows in the entire transport network based on equilibrium flows.The total emission level is used to reflect the impacts of distance tolls on the environment.With these two indexes/measurements for the efficiency,equity and environmental issues as well as the DUE model,a multi-objective bi-level programming model is then developed to determine optimal distance tolls.The multi-objective model is converted to a single level model using the goal programming.A genetic algorithm(GA) is adopted to determine solutions.Finally,a numerical example is presented to verify the methodology.

Design and optimization of a semi-active suspension system for railway applications

The present work focused on the application of innovative damping technologies in order to improve railway vehicle performances in terms of dynamic stability and comfort. As a benchmark case-study, the secondary sus- pension stage was selected and different control techniques were investigated, such as skyhook, dynamic compensation, and sliding mode control. The final aim was to investigate which control schemes are suitable for optimal exploitation of the non-linear behavior of the actuators. The performance improvement achieved by adoption of the semi-active dampers on a standard high-speed train was evaluated in terms of passenger comfort. Different control strategies have been investigated by comparing a simple SISO （single input single output） regulator based on the skyhook damper ap- proach with a centralized regulator. The centralized regulator allows for the estimation of a near optimal set of control forces that minimize car-body accelerations with respect to constraints imposed by limited performance of semi-active actuators. Simulation results show that best results is obtained using a mixed approach that considers the simultaneous applications of model based and feedback compensation control terms.