Fixed-Time Gradient Flows for Solving Constrained Optimization: A Unified Approach

The accelerated method in solving optimization problems has always been an absorbing topic.Based on the fixedtime(FxT)stability of nonlinear dynamical systems,we provide a unified approach for designing FxT gradient flows(FxTGFs).First,a general class of nonlinear functions in designing FxTGFs is provided.A unified method for designing first-order FxTGFs is shown under Polyak-Łjasiewicz inequality assumption,a weaker condition than strong convexity.When there exist both bounded and vanishing disturbances in the gradient flow,a specific class of nonsmooth robust FxTGFs with disturbance rejection is presented.Under the strict convexity assumption,Newton-based FxTGFs is given and further extended to solve time-varying optimization.Besides,the proposed FxTGFs are further used for solving equation-constrained optimization.Moreover,an FxT proximal gradient flow with a wide range of parameters is provided for solving nonsmooth composite optimization.To show the effectiveness of various FxTGFs,the static regret analyses for several typical FxTGFs are also provided in detail.Finally,the proposed FxTGFs are applied to solve two network problems,i.e.,the network consensus problem and solving a system linear equations,respectively,from the perspective of optimization.Particularly,by choosing component-wisely sign-preserving functions,these problems can be solved in a distributed way,which extends the existing results.The accelerated convergence and robustness of the proposed FxTGFs are validated in several numerical examples stemming from practical applications.

Improved Differential Evolution with Shrinking Space Technique for Constrained Optimization

Most of the current evolutionary algorithms for constrained optimization algorithm are low computational efficiency. In order to improve efficiency, an improved differential evolution with shrinking space technique and adaptive trade-off model, named ATMDE, is proposed to solve constrained optimization problems. The proposed ATMDE algorithm employs an improved differential evolution as the search optimizer to generate new offspring individuals into evolutionary population. For the con- straints, the adaptive trade-off model as one of the most important constraint-handling techniques is employed to select better individuals to retain into the next population, which could effectively handle multiple constraints. Then the shrinking space technique is designed to shrink the search region according to feedback information in order to improve computational efficiency without losing accuracy. The improved DE algorithm introduces three different mutant strategies to generate different offspring into evo- lutionary population. Moreover, a new mutant strategy called ＂DE/rand/best/l＂ is constructed to generate new individuals according to the feasibility proportion ofcurrent population. Finally, the effectiveness of the pro- posed method is verified by a suite of benchmark functions and practical engineering problems. This research presents a constrained evolutionary algorithm with high efficiency and accuracy for constrained optimization problems.

Multiobjective evolutionary algorithm for dynamic nonlinear constrained optimization problems

A new method to solve dynamic nonlinear constrained optimization problems （DNCOP） is proposed. First, the time （environment） variable period of DNCOP is divided into several equal subperiods. In each subperiod, the DNCOP is approximated by a static nonlinear constrained optimization problem （SNCOP）. Second, for each SNCOP, inspired by the idea of multiobjective optimization, it is transformed into a static bi-objective optimization problem. As a result, the original DNCOP is approximately transformed into several static bi-objective optimization problems. Third, a new multiobjective evolutionary algorithm is proposed based on a new selection operator and an improved nonuniformity mutation operator. The simulation results indicate that the proposed algorithm is effective for DNCOP.

Feasible SQP Descent Method for Inequality Constrained Optimization Problems and Its Convergence

In this paper,the new SQP feasible descent algorithm for nonlinear constrained optimization problems presented,and under weaker conditions of relative,we proofed the new method still possesses global convergence and its strong convergence.The numerical results illustrate that the new methods are valid.

A TRUST REGION METHOD WITH A CONIC MODEL FOR NONLINEARLY CONSTRAINED OPTIMIZATION

Trust region methods are powerful and effective optimization methods. The conic model method is a new type of method with more information available at each iteration than standard quadratic-based methods. The advantages of the above two methods can be combined to form a more powerful method for constrained optimization. The trust region subproblem of our method is to minimize a conic function subject to the linearized constraints and trust region bound. At the same time, the new algorithm still possesses robust global properties. The global convergence of the new algorithm under standard conditions is established.

Remarks on a benchmark nonlinear constrained optimization problem

Remarks on a benchmark nonlinear constrained optimization problem are made. Due to a citation error, two absolutely different results for the benchmark problem are obtained by independent researchers. Parallel simulated annealing using simplex method is employed in our study to solve the benchmark nonlinear constrained problem with mistaken formula and the best-known solution is obtained, whose optimality is testified by the Kuhn Tucker conditions.

An improved flexible tolerance method for solving nonlinear constrained optimization problems:Application in mass integration

This paper proposes the use of the flexible tolerance method(FTM) modified with adaptive Nelder–Mead parameters and barrier to solve constrained optimization problems. The problems used to analyze the performance of the methods were taken from G-Suite functions, and the methods with the best performance were applied in mass integration problems. Four methods were proposed:(1) flexible tolerance method(FTM) using adaptive parameters(FTMA),(2) flexible tolerance method with scaling(FTMS) and with adaptive parameters(FTMAS),(3) FTMS including the barrier modification(MFTMS) and(4) MFTMS hybridized with PSO(MFTMS-PSO). The success rates of these methods were 100%(MFTMS), 85%(MFTMS-PSO), 40%(FTMAS) and 30%(FTMA).Numerical experiments indicated that the MFTMS could efficiently and reliably improve the accuracy of global optima. In mass integration, the method was able, from current process situation, to reach the optimum process configuration that includes integration issues, which was not possible using FTM in its standard formulation. The hybridization of FTMS with PSO(without barrier), FTMS-PSO, was also able to solve mass integration problems efficiently.

Convergence analysis of a nonlinear Lagrange algorithm for general nonlinear constrained optimization problems

The convergence analysis of a nonlinear Lagrange algorithm for solving nonlinear constrained optimization problems with both inequality and equality constraints is explored in detail. The estimates for the derivatives of the multiplier mapping and the solution mapping of the proposed algorithm are discussed via the technique of the singular value decomposition of matrix. Based on the estimates, the local convergence results and the rate of convergence of the algorithm are presented when the penalty parameter is less than a threshold under a set of suitable conditions on problem functions. Furthermore, the condition number of the Hessian of the nonlinear Lagrange function with respect to the decision variables is analyzed, which is closely related to efficiency of the algorithm. Finally, the preliminary numericM results for several typical test problems are reported.

Shape modification of Bézier curves by constrained optimization

The Bézier curve is one of the most commonly used parametric curves in CAGD and Computer Graphics and has many good properties for shape design. Developing more convenient techniques for designing and modifying Bézier curve is an im- portant problem, and is also an important research issue in CAD/CAM and NC technology fields. This work investigates the optimal shape modification of Bézier curves by geometric constraints. This paper presents a new method by constrained optimi- zation based on changing the control points of the curves. By this method, the authors modify control points of the original Bézier curves to satisfy the given constraints and modify the shape of the curves optimally. Practical examples are also given.

A new evolutionary algorithm for constrained optimization problems

To solve single-objective constrained optimization problems,a new population-based evolutionary algorithm with elite strategy(PEAES) is proposed with the concept of single and multi-objective optimization.Constrained functions are combined to be an objective function.During the evolutionary process,the current optimal solution is found and treated as the reference point to divide the population into three sub-populations:one feasible and two infeasible ones.Different evolutionary operations of single or multi-objective optimization are respectively performed in each sub-population with elite strategy.Thirteen famous benchmark functions are selected to evaluate the performance of PEAES in comparison of other three optimization methods.The results show the proposed method is valid in efficiency,precision and probability for solving single-objective constrained optimization problems.

A CLASS OF TRUST REGION METHODS FOR LINEAR INEQUALITY CONSTRAINED OPTIMIZATION AND ITS THEORY ANALYSIS:I.ALGORITHM AND GLOBAL CONVERGENCEXIU NAIHUA

A class of trust region methods for solving linear inequality constrained problems is proposed in this paper. It is shown that the algorithm is of global convergence.The algorithm uses a version of the two-sided projection and the strategy of the unconstrained trust region methods. It keeps the good convergence properties of the unconstrained case and has the merits of the projection method. In some sense, our algorithm can be regarded as an extension and improvement of the projected type algorithm.

A SMOOTHING QP-FREE INFEASIBLE METHOD FOR NONLINEAR INEQUALITY CONSTRAINED OPTIMIZATION

In this paper, a smoothing QP-free infeasible method is proposed for nonlinear inequality constrained optimization problems. This iterative method is based on the solution of nonlinear equations which is obtained by the multipliers and the smoothing FisheroBurmeister function for the KKT first-order optimality conditions. Comparing with other QP-free methods, this method does not request the strict feasibility of iteration. In particular, this method is implementable and globally convergent without assuming the strict complementarity condition and the isolatedness of accumulation points. ~rthermore, the gradients of active constraints are not requested to be linearly independent. Preliminary numerical results indicate that this smoothing QP-free infeasible method is quite promising.

A New Unified Path to Smoothing Nonsmooth Exact Penalty Function for the Constrained Optimization

We propose a new unified path to approximately smoothing the nonsmooth exact penalty function in this paper. Based on the new smooth penalty function, we give a penalty algorithm to solve the constrained optimization problem, and discuss the convergence of the algorithm under mild conditions.

A SUPERLINEARLY AND QUADRATICALLY CONVERGENT SQP TYPE FEASIBLE METHOD FOR CONSTRAINED OPTIMIZATION

A new SQP type feasible method for inequality constrained optimization is presented,it is a combination of a master algorithm and an auxiliary algorithm which is taken only in finite iterations.The directions of the master algorithm are generated by only one quadratic programming, and its step\|size is always one, the directions of the auxiliary algorithm are new “second\|order” feasible descent. Under suitable assumptions,the algorithm is proved to possess global and strong convergence, superlinear and quadratic convergence.

A MIXED SUPERLINEARLY CONVERGENT ALGORITHM WITH NONMONOTONE SEARCH FOR CONSTRAINED OPTIMIZATIONS

In the paper, a new mixed algorithm combined with schemes of nonmonotone line search, the systems of linear equations for higher order modification and sequential quadratic programming for constrained optimizations is presented. Under some weaker assumptions,without strict complementary condition, the algorithm is globally and superlinearly convergent.

Surrogate-assisted differential evolution using manifold learning-based sampling for highdimensional expensive constrained optimization problems

To address the challenges of high-dimensional constrained optimization problems with expensive simulation models,a Surrogate-Assisted Differential Evolution using Manifold Learning-based Sampling(SADE-MLS)is proposed.In SADE-MLS,differential evolution operators are executed to generate numerous high-dimensional candidate points.To alleviate the curse of dimensionality,a Manifold Learning-based Sampling(MLS)mechanism is developed to explore the high-dimensional design space effectively.In MLS,the intrinsic dimensionality of the candidate points is determined by a maximum likelihood estimator.Then,the candidate points are mapped into a low-dimensional space using the dimensionality reduction technique,which can avoid significant information loss during dimensionality reduction.Thus,Kriging surrogates are constructed in the low-dimensional space to predict the responses of the mapped candidate points.The candidate points with high constrained expected improvement values are selected for global exploration.Moreover,the local search process assisted by radial basis function and differential evolution is performed to exploit the design space efficiently.Several numerical benchmarks are tested to compare SADE-MLS with other algorithms.Finally,SADE-MLS is successfully applied to a solid rocket motor multidisciplinary optimization problem and a re-entry vehicle aerodynamic optimization problem,with the total impulse and lift to drag ratio being increased by 32.7%and 35.5%,respec-tively.The optimization results demonstrate the practicality and effectiveness of the proposed method in real engineering practices.

Even Search in a Promising Region for Constrained Multi-Objective Optimization

In recent years, a large number of approaches to constrained multi-objective optimization problems(CMOPs) have been proposed, focusing on developing tweaked strategies and techniques for handling constraints. However, an overly finetuned strategy or technique might overfit some problem types,resulting in a lack of versatility. In this article, we propose a generic search strategy that performs an even search in a promising region. The promising region, determined by obtained feasible non-dominated solutions, possesses two general properties.First, the constrained Pareto front(CPF) is included in the promising region. Second, as the number of feasible solutions increases or the convergence performance(i.e., approximation to the CPF) of these solutions improves, the promising region shrinks. Then we develop a new strategy named even search,which utilizes the non-dominated solutions to accelerate convergence and escape from local optima, and the feasible solutions under a constraint relaxation condition to exploit and detect feasible regions. Finally, a diversity measure is adopted to make sure that the individuals in the population evenly cover the valuable areas in the promising region. Experimental results on 45 instances from four benchmark test suites and 14 real-world CMOPs have demonstrated that searching evenly in the promising region can achieve competitive performance and excellent versatility compared to 11 most state-of-the-art methods tailored for CMOPs.

Constrained Multi-Objective Optimization With Deep Reinforcement Learning Assisted Operator Selection

Solving constrained multi-objective optimization problems with evolutionary algorithms has attracted considerable attention.Various constrained multi-objective optimization evolutionary algorithms(CMOEAs)have been developed with the use of different algorithmic strategies,evolutionary operators,and constraint-handling techniques.The performance of CMOEAs may be heavily dependent on the operators used,however,it is usually difficult to select suitable operators for the problem at hand.Hence,improving operator selection is promising and necessary for CMOEAs.This work proposes an online operator selection framework assisted by Deep Reinforcement Learning.The dynamics of the population,including convergence,diversity,and feasibility,are regarded as the state;the candidate operators are considered as actions;and the improvement of the population state is treated as the reward.By using a Q-network to learn a policy to estimate the Q-values of all actions,the proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state and thereby improve the algorithmic performance.The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems.The experimental results reveal that the proposed Deep Reinforcement Learning-assisted operator selection significantly improves the performance of these CMOEAs and the resulting algorithm obtains better versatility compared to nine state-of-the-art CMOEAs.

Constraints Separation Based Evolutionary Multitasking for Constrained Multi-Objective Optimization Problems

Constrained multi-objective optimization problems(CMOPs)generally contain multiple constraints,which not only form multiple discrete feasible regions but also reduce the size of optimal feasible regions,thus they propose serious challenges for solvers.Among all constraints,some constraints are highly correlated with optimal feasible regions;thus they can provide effective help to find feasible Pareto front.However,most of the existing constrained multi-objective evolutionary algorithms tackle constraints by regarding all constraints as a whole or directly ignoring all constraints,and do not consider judging the relations among constraints and do not utilize the information from promising single constraints.Therefore,this paper attempts to identify promising single constraints and utilize them to help solve CMOPs.To be specific,a CMOP is transformed into a multitasking optimization problem,where multiple auxiliary tasks are created to search for the Pareto fronts that only consider a single constraint respectively.Besides,an auxiliary task priority method is designed to identify and retain some high-related auxiliary tasks according to the information of relative positions and dominance relationships.Moreover,an improved tentative method is designed to find and transfer useful knowledge among tasks.Experimental results on three benchmark test suites and 11 realworld problems with different numbers of constraints show better or competitive performance of the proposed method when compared with eight state-of-the-art peer methods.

Evolutionary Multitasking With Global and Local Auxiliary Tasks for Constrained Multi-Objective Optimization

Constrained multi-objective optimization problems(CMOPs) include the optimization of objective functions and the satisfaction of constraint conditions, which challenge the solvers.To solve CMOPs, constrained multi-objective evolutionary algorithms(CMOEAs) have been developed. However, most of them tend to converge into local areas due to the loss of diversity. Evolutionary multitasking(EMT) is new model of solving complex optimization problems, through the knowledge transfer between the source task and other related tasks. Inspired by EMT, this paper develops a new EMT-based CMOEA to solve CMOPs, in which the main task, a global auxiliary task, and a local auxiliary task are created and optimized by one specific population respectively. The main task focuses on finding the feasible Pareto front(PF), and global and local auxiliary tasks are used to respectively enhance global and local diversity. Moreover, the global auxiliary task is used to implement the global search by ignoring constraints, so as to help the population of the main task pass through infeasible obstacles. The local auxiliary task is used to provide local diversity around the population of the main task, so as to exploit promising regions. Through the knowledge transfer among the three tasks, the search ability of the population of the main task will be significantly improved. Compared with other state-of-the-art CMOEAs, the experimental results on three benchmark test suites demonstrate the superior or competitive performance of the proposed CMOEA.