General Lyapunov Stability and Its Application to Time-Varying Convex Optimization

In this article, a general Lyapunov stability theory of nonlinear systems is put forward and it contains asymptotic/finite-time/fast finite-time/fixed-time stability. Especially, a more accurate estimate of the settling-time function is exhibited for fixedtime stability, and it is still extraneous to the initial conditions.This can be applied to obtain less conservative convergence time of the practical systems without the information of the initial conditions. As an application, the given fixed-time stability theorem is used to resolve time-varying(TV) convex optimization problem.By the Newton's method, two classes of new dynamical systems are constructed to guarantee that the solution of the dynamic system can track to the optimal trajectory of the unconstrained and equality constrained TV convex optimization problems in fixed time, respectively. Without the exact knowledge of the time derivative of the cost function gradient, a fixed-time dynamical non-smooth system is established to overcome the issue of robust TV convex optimization. Two examples are provided to illustrate the effectiveness of the proposed TV convex optimization algorithms. Subsequently, the fixed-time stability theory is extended to the theories of predefined-time/practical predefined-time stability whose bound of convergence time can be arbitrarily given in advance, without tuning the system parameters. Under which, TV convex optimization problem is solved. The previous two examples are used to demonstrate the validity of the predefined-time TV convex optimization algorithms.

Distributed Stochastic Optimization with Compression for Non-Strongly Convex Objectives

We are investigating the distributed optimization problem,where a network of nodes works together to minimize a global objective that is a finite sum of their stored local functions.Since nodes exchange optimization parameters through the wireless network,large-scale training models can create communication bottlenecks,resulting in slower training times.To address this issue,CHOCO-SGD was proposed,which allows compressing information with arbitrary precision without reducing the convergence rate for strongly convex objective functions.Nevertheless,most convex functions are not strongly convex(such as logistic regression or Lasso),which raises the question of whether this algorithm can be applied to non-strongly convex functions.In this paper,we provide the first theoretical analysis of the convergence rate of CHOCO-SGD on non-strongly convex objectives.We derive a sufficient condition,which limits the fidelity of compression,to guarantee convergence.Moreover,our analysis demonstrates that within the fidelity threshold,this algorithm can significantly reduce transmission burden while maintaining the same convergence rate order as its no-compression equivalent.Numerical experiments further validate the theoretical findings by demonstrating that CHOCO-SGD improves communication efficiency and keeps the same convergence rate order simultaneously.And experiments also show that the algorithm fails to converge with low compression fidelity and in time-varying topologies.Overall,our study offers valuable insights into the potential applicability of CHOCO-SGD for non-strongly convex objectives.Additionally,we provide practical guidelines for researchers seeking to utilize this algorithm in real-world scenarios.

Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

The Nesterov accelerated dynamical approach serves as an essential tool for addressing convex optimization problems with accelerated convergence rates.Most previous studies in this field have primarily concentrated on unconstrained smooth con-vex optimization problems.In this paper,on the basis of primal-dual dynamical approach,Nesterov accelerated dynamical approach,projection operator and directional gradient,we present two accelerated primal-dual projection neurodynamic approaches with time scaling to address convex optimization problems with smooth and nonsmooth objective functions subject to linear and set constraints,which consist of a second-order ODE(ordinary differential equation)or differential conclusion system for the primal variables and a first-order ODE for the dual vari-ables.By satisfying specific conditions for time scaling,we demonstrate that the proposed approaches have a faster conver-gence rate.This only requires assuming convexity of the objective function.We validate the effectiveness of our proposed two accel-erated primal-dual projection neurodynamic approaches through numerical experiments.

On the Linear Convergence of the Approximate Proximal Splitting Method for Non-smooth Convex Optimization

Consider the problem of minimizing the sum of two convex functions,one being smooth and the other non-smooth.In this paper,we introduce a general class of approximate proximal splitting(APS)methods for solving such minimization problems.Methods in the APS class include many well-known algorithms such as the proximal splitting method,the block coordinate descent method(BCD),and the approximate gradient projection methods for smooth convex optimization.We establish the linear convergence of APS methods under a local error bound assumption.Since the latter is known to hold for compressive sensing and sparse group LASSO problems,our analysis implies the linear convergence of the BCD method for these problems without strong convexity assumption.

Cooperative User-Scheduling and Resource Allocation Optimization for Intelligent Reflecting Surface Enhanced LEO Satellite Communication

Lower Earth Orbit(LEO) satellite becomes an important part of complementing terrestrial communication due to its lower orbital altitude and smaller propagation delay than Geostationary satellite. However, the LEO satellite communication system cannot meet the requirements of users when the satellite-terrestrial link is blocked by obstacles. To solve this problem, we introduce Intelligent reflect surface(IRS) for improving the achievable rate of terrestrial users in LEO satellite communication. We investigated joint IRS scheduling, user scheduling, power and bandwidth allocation(JIRPB) optimization algorithm for improving LEO satellite system throughput.The optimization problem of joint user scheduling and resource allocation is formulated as a non-convex optimization problem. To cope with this problem, the nonconvex optimization problem is divided into resource allocation optimization sub-problem and scheduling optimization sub-problem firstly. Second, we optimize the resource allocation sub-problem via alternating direction multiplier method(ADMM) and scheduling sub-problem via Lagrangian dual method repeatedly.Third, we prove that the proposed resource allocation algorithm based ADMM approaches sublinear convergence theoretically. Finally, we demonstrate that the proposed JIRPB optimization algorithm improves the LEO satellite communication system throughput.

A novel image segmentation approach for wood plate surface defect classification through convex optimization

Detection of wood plate surface defects using image processing is a complicated problem in the forest industry as the image of the wood surface contains different kinds of defects. In order to obtain complete defect images, we used convex optimization(CO) with different weights as a pretreatment method for smoothing and the Otsu segmentation method to obtain the target defect area images. Structural similarity(SSIM) results between original image and defect image were calculated to evaluate the performance of segmentation with different convex optimization weights. The geometric and intensity features of defects were extracted before constructing a classification and regression tree(CART) classifier. The average accuracy of the classifier is 94.1% with four types of defects on Xylosma congestum wood plate surface: pinhole, crack,live knot and dead knot. Experimental results showed that CO can save the edge of target defects maximally, SSIM can select the appropriate weight for CO, and the CART classifier appears to have the advantages of good adaptability and high classification accuracy.

A new primal-dual interior-point algorithm for convex quadratic optimization

In this paper, a new primal-dual interior-point algorithm for convex quadratic optimization （CQO） based on a kernel function is presented. The proposed function has some properties that are easy for checking. These properties enable us to improve the polynomial complexity bound of a large-update interior-point method （IPM） to O（√n log nlog n/e）, which is the currently best known polynomial complexity bound for the algorithm with the large-update method. Numerical tests were conducted to investigate the behavior of the algorithm with different parameters p, q and θ, where p is the growth degree parameter, q is the barrier degree of the kernel function and θ is the barrier update parameter.

Learning Convex Optimization Models

A convex optimization model predicts an output from an input by solving a convex optimization problem.The class of convex optimization models is large,and includes as special cases many well-known models like linear and logistic regression.We propose a heuristic for learning the parameters in a convex optimization model given a dataset of input-output pairs,using recently developed methods for differentiating the solution of a convex optimization problem with respect to its parameters.We describe three general classes of convex optimization models,maximum a posteriori(MAP)models,utility maximization models,and agent models,and present a numerical experiment for each.

An Overview of Sequential Approximation in Topology Optimization of Continuum Structure

This paper offers an extensive overview of the utilization of sequential approximate optimization approaches in the context of numerically simulated large-scale continuum structures.These structures,commonly encountered in engineering applications,often involve complex objective and constraint functions that cannot be readily expressed as explicit functions of the design variables.As a result,sequential approximation techniques have emerged as the preferred strategy for addressing a wide array of topology optimization challenges.Over the past several decades,topology optimization methods have been advanced remarkably and successfully applied to solve engineering problems incorporating diverse physical backgrounds.In comparison to the large-scale equation solution,sensitivity analysis,graphics post-processing,etc.,the progress of the sequential approximation functions and their corresponding optimizersmake sluggish progress.Researchers,particularly novices,pay special attention to their difficulties with a particular problem.Thus,this paper provides an overview of sequential approximation functions,related literature on topology optimization methods,and their applications.Starting from optimality criteria and sequential linear programming,the other sequential approximate optimizations are introduced by employing Taylor expansion and intervening variables.In addition,recent advancements have led to the emergence of approaches such as Augmented Lagrange,sequential approximate integer,and non-gradient approximation are also introduced.By highlighting real-world applications and case studies,the paper not only demonstrates the practical relevance of these methods but also underscores the need for continued exploration in this area.Furthermore,to provide a comprehensive overview,this paper offers several novel developments that aim to illuminate potential directions for future research.

Optimization in Machine Learning:a Distribution-Space Approach

We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space,but with a non-convex constraint set introduced by model parameterization.This observation allows us to repose such problems via a suitable relaxation as convex optimization problems in the space of distributions over the training parameters.We derive some simple relationships between the distribution-space problem and the original problem,e.g.,a distribution-space solution is at least as good as a solution in the original space.Moreover,we develop a numerical algorithm based on mixture distributions to perform approximate optimization directly in the distribution space.Consistency of this approximation is established and the numerical efficacy of the proposed algorithm is illustrated in simple examples.In both theory and practice,this formulation provides an alternative approach to large-scale optimization in machine learning.

Multiconstraint adaptive three-dimensional guidance law using convex optimization

The traditional guidance law only guarantees the accuracy of attacking a target. However, the look angle and acceleration constraints are indispensable in applications. A new adaptive three-dimensional proportional navigation(PN) guidance law is proposed based on convex optimization. Decomposition of the three-dimensional space is carried out to establish threedimensional kinematic engagements. The constraints and the performance index are disposed by using the convex optimization method. PN guidance gains can be obtained by solving the optimization problem. This solution is more rapid and programmatic than the traditional method and provides a foundation for future online guidance methods, which is of great value for engineering applications.

Computation of Peak Output for Inputs Restricted in L_2 and L_∞ Norms Using Finite Difference Schemes and Convex Optimization

Control systems designed by the principle of matching gives rise to problems of evaluating the peak output. This paper proposes a practical method for computing the peak output of linear time-invariant and non-anticipative systems for a class of possible sets that are characterized with many bounding conditions on the two- and/or the infinity-norms of the inputs and their derivatives. The original infinite-dimensional convex optimization problem is approximated as a large-scale convex programme defined in a Euclidean space, which are associated with sparse matrices and thus can be solved efficiently in practice. The numerical results show that the method performs satisfactorily, and that using a possible set with many bounding conditions can help to reduce the design conservatism and thereby yield a better match.

NONCOMPACT INFINITE OPTIMIZATION AND EQUILIBRIA OF CONSTRAINED GAMES IN GENERALIZED CONVEX SPACES

By applying a new existence theorem of quasi-equilibrium problems due to the author, some existence theorems of solutions for noncompact infinite optimization problems and noncompact constrained game problems are proved in generalized convex spaces without linear structure. These theorems improve and generalize a number of important results in recent literature.

Performance Analysis of Sparse Array based Massive MIMO via Joint Convex Optimization

Massive multiple-input multiple-output(MIMO)technology enables higher data rate transmission in the future mobile communications.However,exploiting a large number of antenna elements at base station(BS)makes effective implementation of massive MIMO challenging,due to the size and weight limits of the masssive MIMO that are located on each BS.Therefore,in order to miniaturize the massive MIMO,it is crucial to reduce the number of antenna elements via effective methods such as sparse array synthesis.In this paper,a multiple-pattern synthesis is considered towards convex optimization(CO).The joint convex optimization(JCO)based synthesis is proposed to construct a codebook for beamforming.Then,a criterion containing multiple constraints is developed,in which the sparse array is required to fullfill all constraints.Finally,extensive evaluations are performed under realistic simulation settings.The results show that with the same number of antenna elements,sparse array using the proposed JCO-based synthesis outperforms not only the uniform array,but also the sparse array with the existing CO-based synthesis method.Furthermore,with a half of the number of antenna elements that on the uniform array,the performance of the JCO-based sparse array approaches to that of the uniform array.

A new primal-dual path-following interior-point algorithm for linearly constrained convex optimization

In this paper, a primal-dual path-following interior-point algorithm for linearly constrained convex optimization(LCCO) is presented.The algorithm is based on a new technique for finding a class of search directions and the strategy of the central path.At each iteration, only full-Newton steps are used.Finally, the favorable polynomial complexity bound for the algorithm with the small-update method is deserved, namely, O(√n log n /ε).

Stabilization of linear time-varying systems with state and input constraints using convex optimization

The stabilization problem of linear time-varying systems with both state and input constraints is considered. Sufficient conditions for the existence of the solution to this problem are derived and a gain-switched（gain-scheduled） state feedback control scheme is built to stabilize the constrained timevarying system. The design problem is transformed to a series of convex feasibility problems which can be solved efficiently. A design example is given to illustrate the effect of the proposed algorithm.

CHARACTERIZATION OF EFFICIENT SOLUTIONS FOR MULTI-OBJECTIVE OPTIMIZATION PROBLEMS INVOLVING SEMI-STRONG AND GENERALIZED SEMI-STRONG E-CONVEXITY

The authors of this article are interested in characterization of efficient solutions for special classes of problems. These classes consider semi-strong E-convexity of involved functions. Sufficient and necessary conditions for a feasible solution to be an efficient or properly efficient solution are obtained.

The Application of Convex Optimization to Load Optimal Allocation in a Single Workstation System

The service quality of a workstation depends mainly on its service load, ifnot taking into account all kinds of devices' break-downs. In this article, an optimization modelwith inequality constraints is proposed, which aims to minimize the service load. A noveltransformation of optimization variables is also devised and the constraints are properly combinedso as to make this model into a convex one, whose corresponding Lagrange function and the KKTconditions are established afterwards. The interior-point method for convex optimization ispresented here as an efficient computation tool. Finally, this model is evaluated by a real example,from which conclusions are reached that the interior-point method possesses advantages such asfaster convergeoce and fewer iterations and it is possible to make complicated nonlinearoptimization problems exhibit convexity so as to obtain the optimum.

An Introduction to Convex Optimization Theory in Communication Signals Recognition

In this paper,convex optimization theory is introduced into the recognition of communication signals. The detailed content contains three parts. The first part gives a survey of basic concepts,main technology and recognition model of convex optimization theory. Special emphasis is placed on how to set up the new recognition model of communication signals with multisensor reports. The second part gives the solution method of the recognition model,which is called Logarithmic Penalty Barrier Function. The last part gives several numeric simulations,in contrast to D-S evidence inference method,this new method can also generate reasonable recognition results. Moreover,this new method can deal with the form of sensor reports which is more general than that allowed by the D-S evidence inference method,and it has much lower computation complexity than that of D-S evidence inference method. In addition,this new method has better recognition result,stronger anti-interference and robustness. Therefore,the convex optimization methods can be widely used in the recognition of communication signals.