Fixed-Time Gradient Flows for Solving Constrained Optimization: A Unified Approach

The accelerated method in solving optimization problems has always been an absorbing topic.Based on the fixedtime(FxT)stability of nonlinear dynamical systems,we provide a unified approach for designing FxT gradient flows(FxTGFs).First,a general class of nonlinear functions in designing FxTGFs is provided.A unified method for designing first-order FxTGFs is shown under Polyak-Łjasiewicz inequality assumption,a weaker condition than strong convexity.When there exist both bounded and vanishing disturbances in the gradient flow,a specific class of nonsmooth robust FxTGFs with disturbance rejection is presented.Under the strict convexity assumption,Newton-based FxTGFs is given and further extended to solve time-varying optimization.Besides,the proposed FxTGFs are further used for solving equation-constrained optimization.Moreover,an FxT proximal gradient flow with a wide range of parameters is provided for solving nonsmooth composite optimization.To show the effectiveness of various FxTGFs,the static regret analyses for several typical FxTGFs are also provided in detail.Finally,the proposed FxTGFs are applied to solve two network problems,i.e.,the network consensus problem and solving a system linear equations,respectively,from the perspective of optimization.Particularly,by choosing component-wisely sign-preserving functions,these problems can be solved in a distributed way,which extends the existing results.The accelerated convergence and robustness of the proposed FxTGFs are validated in several numerical examples stemming from practical applications.

Improved Differential Evolution with Shrinking Space Technique for Constrained Optimization

Most of the current evolutionary algorithms for constrained optimization algorithm are low computational efficiency. In order to improve efficiency, an improved differential evolution with shrinking space technique and adaptive trade-off model, named ATMDE, is proposed to solve constrained optimization problems. The proposed ATMDE algorithm employs an improved differential evolution as the search optimizer to generate new offspring individuals into evolutionary population. For the con- straints, the adaptive trade-off model as one of the most important constraint-handling techniques is employed to select better individuals to retain into the next population, which could effectively handle multiple constraints. Then the shrinking space technique is designed to shrink the search region according to feedback information in order to improve computational efficiency without losing accuracy. The improved DE algorithm introduces three different mutant strategies to generate different offspring into evo- lutionary population. Moreover, a new mutant strategy called ＂DE/rand/best/l＂ is constructed to generate new individuals according to the feasibility proportion ofcurrent population. Finally, the effectiveness of the pro- posed method is verified by a suite of benchmark functions and practical engineering problems. This research presents a constrained evolutionary algorithm with high efficiency and accuracy for constrained optimization problems.

Multiobjective evolutionary algorithm for dynamic nonlinear constrained optimization problems

A new method to solve dynamic nonlinear constrained optimization problems （DNCOP） is proposed. First, the time （environment） variable period of DNCOP is divided into several equal subperiods. In each subperiod, the DNCOP is approximated by a static nonlinear constrained optimization problem （SNCOP）. Second, for each SNCOP, inspired by the idea of multiobjective optimization, it is transformed into a static bi-objective optimization problem. As a result, the original DNCOP is approximately transformed into several static bi-objective optimization problems. Third, a new multiobjective evolutionary algorithm is proposed based on a new selection operator and an improved nonuniformity mutation operator. The simulation results indicate that the proposed algorithm is effective for DNCOP.

A TRUST REGION METHOD WITH A CONIC MODEL FOR NONLINEARLY CONSTRAINED OPTIMIZATION

Trust region methods are powerful and effective optimization methods. The conic model method is a new type of method with more information available at each iteration than standard quadratic-based methods. The advantages of the above two methods can be combined to form a more powerful method for constrained optimization. The trust region subproblem of our method is to minimize a conic function subject to the linearized constraints and trust region bound. At the same time, the new algorithm still possesses robust global properties. The global convergence of the new algorithm under standard conditions is established.

Remarks on a benchmark nonlinear constrained optimization problem

Remarks on a benchmark nonlinear constrained optimization problem are made. Due to a citation error, two absolutely different results for the benchmark problem are obtained by independent researchers. Parallel simulated annealing using simplex method is employed in our study to solve the benchmark nonlinear constrained problem with mistaken formula and the best-known solution is obtained, whose optimality is testified by the Kuhn Tucker conditions.

An improved flexible tolerance method for solving nonlinear constrained optimization problems:Application in mass integration

This paper proposes the use of the flexible tolerance method(FTM) modified with adaptive Nelder–Mead parameters and barrier to solve constrained optimization problems. The problems used to analyze the performance of the methods were taken from G-Suite functions, and the methods with the best performance were applied in mass integration problems. Four methods were proposed:(1) flexible tolerance method(FTM) using adaptive parameters(FTMA),(2) flexible tolerance method with scaling(FTMS) and with adaptive parameters(FTMAS),(3) FTMS including the barrier modification(MFTMS) and(4) MFTMS hybridized with PSO(MFTMS-PSO). The success rates of these methods were 100%(MFTMS), 85%(MFTMS-PSO), 40%(FTMAS) and 30%(FTMA).Numerical experiments indicated that the MFTMS could efficiently and reliably improve the accuracy of global optima. In mass integration, the method was able, from current process situation, to reach the optimum process configuration that includes integration issues, which was not possible using FTM in its standard formulation. The hybridization of FTMS with PSO(without barrier), FTMS-PSO, was also able to solve mass integration problems efficiently.

Convergence analysis of a nonlinear Lagrange algorithm for general nonlinear constrained optimization problems

The convergence analysis of a nonlinear Lagrange algorithm for solving nonlinear constrained optimization problems with both inequality and equality constraints is explored in detail. The estimates for the derivatives of the multiplier mapping and the solution mapping of the proposed algorithm are discussed via the technique of the singular value decomposition of matrix. Based on the estimates, the local convergence results and the rate of convergence of the algorithm are presented when the penalty parameter is less than a threshold under a set of suitable conditions on problem functions. Furthermore, the condition number of the Hessian of the nonlinear Lagrange function with respect to the decision variables is analyzed, which is closely related to efficiency of the algorithm. Finally, the preliminary numericM results for several typical test problems are reported.

A new evolutionary algorithm for constrained optimization problems

To solve single-objective constrained optimization problems,a new population-based evolutionary algorithm with elite strategy(PEAES) is proposed with the concept of single and multi-objective optimization.Constrained functions are combined to be an objective function.During the evolutionary process,the current optimal solution is found and treated as the reference point to divide the population into three sub-populations:one feasible and two infeasible ones.Different evolutionary operations of single or multi-objective optimization are respectively performed in each sub-population with elite strategy.Thirteen famous benchmark functions are selected to evaluate the performance of PEAES in comparison of other three optimization methods.The results show the proposed method is valid in efficiency,precision and probability for solving single-objective constrained optimization problems.

A SMOOTHING QP-FREE INFEASIBLE METHOD FOR NONLINEAR INEQUALITY CONSTRAINED OPTIMIZATION

In this paper, a smoothing QP-free infeasible method is proposed for nonlinear inequality constrained optimization problems. This iterative method is based on the solution of nonlinear equations which is obtained by the multipliers and the smoothing FisheroBurmeister function for the KKT first-order optimality conditions. Comparing with other QP-free methods, this method does not request the strict feasibility of iteration. In particular, this method is implementable and globally convergent without assuming the strict complementarity condition and the isolatedness of accumulation points. ~rthermore, the gradients of active constraints are not requested to be linearly independent. Preliminary numerical results indicate that this smoothing QP-free infeasible method is quite promising.

A New Unified Path to Smoothing Nonsmooth Exact Penalty Function for the Constrained Optimization

We propose a new unified path to approximately smoothing the nonsmooth exact penalty function in this paper. Based on the new smooth penalty function, we give a penalty algorithm to solve the constrained optimization problem, and discuss the convergence of the algorithm under mild conditions.

A SUPERLINEARLY AND QUADRATICALLY CONVERGENT SQP TYPE FEASIBLE METHOD FOR CONSTRAINED OPTIMIZATION

A new SQP type feasible method for inequality constrained optimization is presented,it is a combination of a master algorithm and an auxiliary algorithm which is taken only in finite iterations.The directions of the master algorithm are generated by only one quadratic programming, and its step\|size is always one, the directions of the auxiliary algorithm are new “second\|order” feasible descent. Under suitable assumptions,the algorithm is proved to possess global and strong convergence, superlinear and quadratic convergence.

A MIXED SUPERLINEARLY CONVERGENT ALGORITHM WITH NONMONOTONE SEARCH FOR CONSTRAINED OPTIMIZATIONS

In the paper, a new mixed algorithm combined with schemes of nonmonotone line search, the systems of linear equations for higher order modification and sequential quadratic programming for constrained optimizations is presented. Under some weaker assumptions,without strict complementary condition, the algorithm is globally and superlinearly convergent.

Even Search in a Promising Region for Constrained Multi-Objective Optimization

In recent years, a large number of approaches to constrained multi-objective optimization problems(CMOPs) have been proposed, focusing on developing tweaked strategies and techniques for handling constraints. However, an overly finetuned strategy or technique might overfit some problem types,resulting in a lack of versatility. In this article, we propose a generic search strategy that performs an even search in a promising region. The promising region, determined by obtained feasible non-dominated solutions, possesses two general properties.First, the constrained Pareto front(CPF) is included in the promising region. Second, as the number of feasible solutions increases or the convergence performance(i.e., approximation to the CPF) of these solutions improves, the promising region shrinks. Then we develop a new strategy named even search,which utilizes the non-dominated solutions to accelerate convergence and escape from local optima, and the feasible solutions under a constraint relaxation condition to exploit and detect feasible regions. Finally, a diversity measure is adopted to make sure that the individuals in the population evenly cover the valuable areas in the promising region. Experimental results on 45 instances from four benchmark test suites and 14 real-world CMOPs have demonstrated that searching evenly in the promising region can achieve competitive performance and excellent versatility compared to 11 most state-of-the-art methods tailored for CMOPs.

Constrained Multi-Objective Optimization With Deep Reinforcement Learning Assisted Operator Selection

Solving constrained multi-objective optimization problems with evolutionary algorithms has attracted considerable attention.Various constrained multi-objective optimization evolutionary algorithms(CMOEAs)have been developed with the use of different algorithmic strategies,evolutionary operators,and constraint-handling techniques.The performance of CMOEAs may be heavily dependent on the operators used,however,it is usually difficult to select suitable operators for the problem at hand.Hence,improving operator selection is promising and necessary for CMOEAs.This work proposes an online operator selection framework assisted by Deep Reinforcement Learning.The dynamics of the population,including convergence,diversity,and feasibility,are regarded as the state;the candidate operators are considered as actions;and the improvement of the population state is treated as the reward.By using a Q-network to learn a policy to estimate the Q-values of all actions,the proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state and thereby improve the algorithmic performance.The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems.The experimental results reveal that the proposed Deep Reinforcement Learning-assisted operator selection significantly improves the performance of these CMOEAs and the resulting algorithm obtains better versatility compared to nine state-of-the-art CMOEAs.

Constraints Separation Based Evolutionary Multitasking for Constrained Multi-Objective Optimization Problems

Constrained multi-objective optimization problems(CMOPs)generally contain multiple constraints,which not only form multiple discrete feasible regions but also reduce the size of optimal feasible regions,thus they propose serious challenges for solvers.Among all constraints,some constraints are highly correlated with optimal feasible regions;thus they can provide effective help to find feasible Pareto front.However,most of the existing constrained multi-objective evolutionary algorithms tackle constraints by regarding all constraints as a whole or directly ignoring all constraints,and do not consider judging the relations among constraints and do not utilize the information from promising single constraints.Therefore,this paper attempts to identify promising single constraints and utilize them to help solve CMOPs.To be specific,a CMOP is transformed into a multitasking optimization problem,where multiple auxiliary tasks are created to search for the Pareto fronts that only consider a single constraint respectively.Besides,an auxiliary task priority method is designed to identify and retain some high-related auxiliary tasks according to the information of relative positions and dominance relationships.Moreover,an improved tentative method is designed to find and transfer useful knowledge among tasks.Experimental results on three benchmark test suites and 11 realworld problems with different numbers of constraints show better or competitive performance of the proposed method when compared with eight state-of-the-art peer methods.

Evolutionary Multitasking With Global and Local Auxiliary Tasks for Constrained Multi-Objective Optimization

Constrained multi-objective optimization problems(CMOPs) include the optimization of objective functions and the satisfaction of constraint conditions, which challenge the solvers.To solve CMOPs, constrained multi-objective evolutionary algorithms(CMOEAs) have been developed. However, most of them tend to converge into local areas due to the loss of diversity. Evolutionary multitasking(EMT) is new model of solving complex optimization problems, through the knowledge transfer between the source task and other related tasks. Inspired by EMT, this paper develops a new EMT-based CMOEA to solve CMOPs, in which the main task, a global auxiliary task, and a local auxiliary task are created and optimized by one specific population respectively. The main task focuses on finding the feasible Pareto front(PF), and global and local auxiliary tasks are used to respectively enhance global and local diversity. Moreover, the global auxiliary task is used to implement the global search by ignoring constraints, so as to help the population of the main task pass through infeasible obstacles. The local auxiliary task is used to provide local diversity around the population of the main task, so as to exploit promising regions. Through the knowledge transfer among the three tasks, the search ability of the population of the main task will be significantly improved. Compared with other state-of-the-art CMOEAs, the experimental results on three benchmark test suites demonstrate the superior or competitive performance of the proposed CMOEA.

AN SQP ALGORITHM WITH NONMONOTONE LINE SEARCHFOR GENERAL NONLINEAR CONSTRAINED OPTIMIZATION PROBLEM

In this paper, an SQP type algorithm with a new nonmonotone line search technique for general constrained optimization problems is presented. The new algorithm does not have to solve the second order correction subproblems for each iterations, but still can circumvent the so-called Maratos effect. The algorithm's global convergence and superlinear convergent rate have been proved. In addition, we can prove that, after a few iterations, correction subproblems need not be solved, so computation amount of the algorithm will be decreased much more. Numerical experiments show that the new algorithm is effective.

An Improved Feasible QP-free Algorithm for Inequality Constrained Optimization

In this paper, an improved feasible QP-free method is proposed to solve nonlinear inequality constrained optimization problems. Here, a new modified method is presented to obtain the revised feasible descent direction. In view of the computational cost, the most attractive feature of the new algorithm is that only one system of linear equations is required to obtain the revised feasible descent direction. Thereby, per single iteration, it is only necessary to solve three systems of linear equations with the same coefficient matrix. In particular, without the positive definiteness assumption on the Hessian estimate, the proposed algorithm is still global convergence. Under some suitable conditions, the superlinear convergence rate is obtained.

A Subspace Version of the Powell–Yuan Trust-Region Algorithm for Equality Constrained Optimization

This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wang and Yuan(Numer.Math.104:241–269,2006)for the standard trust-region subproblem.Under suitable conditions,it is shown that the trial step obtained from the CDT subproblem is in the subspace spanned by all the gradient vectors of the objective function and of the constraints computed until the current iteration.Based on this observation,a subspace version of the Powell–Yuan trust-region algorithm is proposed for equality constrained optimization problems where the number of constraints is much lower than the number of variables. The convergence analysis is given and numerical results arealso reported.

A relative feasibility degree based approach for constrained optimization problems

Based on the ratio of the size of the feasible region of constraints to the size of the feasible region of a constrained optimization problem,we propose a new constraint handling approach to improve the efficiency of heuristic search methods in solving the constrained optimization problems.In the traditional classification of a solution candidate,it is either a feasible or an infeasible solution.To refine this classification,a new concept about the relative feasibility degree of a solution candidate is proposed to represent the amount by which the 'feasibility' of the solution candidate exceeds that of another candidate.Relative feasibility degree based selection rules are also proposed to enable evolutionary computation techniques to accelerate the search process of reaching a feasible region.In addition,a relative feasibility degree based differential evolution algorithm is derived to solve constraint optimization problems.The proposed approach is tested with nine benchmark problems.Results indicate that our approach is very competitive compared with four existing state-of-the-art techniques,though still sensitive to the intervals of control parameters of the differential evolution.