Evolutionary Optimization Methods for High-Dimensional Expensive Problems:A Survey

Evolutionary computation is a rapidly evolving field and the related algorithms have been successfully used to solve various real-world optimization problems.The past decade has also witnessed their fast progress to solve a class of challenging optimization problems called high-dimensional expensive problems(HEPs).The evaluation of their objective fitness requires expensive resource due to their use of time-consuming physical experiments or computer simulations.Moreover,it is hard to traverse the huge search space within reasonable resource as problem dimension increases.Traditional evolutionary algorithms(EAs)tend to fail to solve HEPs competently because they need to conduct many such expensive evaluations before achieving satisfactory results.To reduce such evaluations,many novel surrogate-assisted algorithms emerge to cope with HEPs in recent years.Yet there lacks a thorough review of the state of the art in this specific and important area.This paper provides a comprehensive survey of these evolutionary algorithms for HEPs.We start with a brief introduction to the research status and the basic concepts of HEPs.Then,we present surrogate-assisted evolutionary algorithms for HEPs from four main aspects.We also give comparative results of some representative algorithms and application examples.Finally,we indicate open challenges and several promising directions to advance the progress in evolutionary optimization algorithms for HEPs.

Constraints Separation Based Evolutionary Multitasking for Constrained Multi-Objective Optimization Problems

Constrained multi-objective optimization problems(CMOPs)generally contain multiple constraints,which not only form multiple discrete feasible regions but also reduce the size of optimal feasible regions,thus they propose serious challenges for solvers.Among all constraints,some constraints are highly correlated with optimal feasible regions;thus they can provide effective help to find feasible Pareto front.However,most of the existing constrained multi-objective evolutionary algorithms tackle constraints by regarding all constraints as a whole or directly ignoring all constraints,and do not consider judging the relations among constraints and do not utilize the information from promising single constraints.Therefore,this paper attempts to identify promising single constraints and utilize them to help solve CMOPs.To be specific,a CMOP is transformed into a multitasking optimization problem,where multiple auxiliary tasks are created to search for the Pareto fronts that only consider a single constraint respectively.Besides,an auxiliary task priority method is designed to identify and retain some high-related auxiliary tasks according to the information of relative positions and dominance relationships.Moreover,an improved tentative method is designed to find and transfer useful knowledge among tasks.Experimental results on three benchmark test suites and 11 realworld problems with different numbers of constraints show better or competitive performance of the proposed method when compared with eight state-of-the-art peer methods.

Enhancing Evolutionary Algorithms With Pattern Mining for Sparse Large-Scale Multi-Objective Optimization Problems

Sparse large-scale multi-objective optimization problems(SLMOPs)are common in science and engineering.However,the large-scale problem represents the high dimensionality of the decision space,requiring algorithms to traverse vast expanse with limited computational resources.Furthermore,in the context of sparse,most variables in Pareto optimal solutions are zero,making it difficult for algorithms to identify non-zero variables efficiently.This paper is dedicated to addressing the challenges posed by SLMOPs.To start,we introduce innovative objective functions customized to mine maximum and minimum candidate sets.This substantial enhancement dramatically improves the efficacy of frequent pattern mining.In this way,selecting candidate sets is no longer based on the quantity of nonzero variables they contain but on a higher proportion of nonzero variables within specific dimensions.Additionally,we unveil a novel approach to association rule mining,which delves into the intricate relationships between non-zero variables.This novel methodology aids in identifying sparse distributions that can potentially expedite reductions in the objective function value.We extensively tested our algorithm across eight benchmark problems and four real-world SLMOPs.The results demonstrate that our approach achieves competitive solutions across various challenges.

Hybrid Hierarchical Particle Swarm Optimization with Evolutionary Artificial Bee Colony Algorithm for Task Scheduling in Cloud Computing

Task scheduling plays a key role in effectively managing and allocating computing resources to meet various computing tasks in a cloud computing environment.Short execution time and low load imbalance may be the challenges for some algorithms in resource scheduling scenarios.In this work,the Hierarchical Particle Swarm Optimization-Evolutionary Artificial Bee Colony Algorithm(HPSO-EABC)has been proposed,which hybrids our presented Evolutionary Artificial Bee Colony(EABC),and Hierarchical Particle Swarm Optimization(HPSO)algorithm.The HPSO-EABC algorithm incorporates both the advantages of the HPSO and the EABC algorithm.Comprehensive testing including evaluations of algorithm convergence speed,resource execution time,load balancing,and operational costs has been done.The results indicate that the EABC algorithm exhibits greater parallelism compared to the Artificial Bee Colony algorithm.Compared with the Particle Swarm Optimization algorithm,the HPSO algorithmnot only improves the global search capability but also effectively mitigates getting stuck in local optima.As a result,the hybrid HPSO-EABC algorithm demonstrates significant improvements in terms of stability and convergence speed.Moreover,it exhibits enhanced resource scheduling performance in both homogeneous and heterogeneous environments,effectively reducing execution time and cost,which also is verified by the ablation experimental.

A Two-Layer Encoding Learning Swarm Optimizer Based on Frequent Itemsets for Sparse Large-Scale Multi-Objective Optimization

Traditional large-scale multi-objective optimization algorithms(LSMOEAs)encounter difficulties when dealing with sparse large-scale multi-objective optimization problems(SLM-OPs)where most decision variables are zero.As a result,many algorithms use a two-layer encoding approach to optimize binary variable Mask and real variable Dec separately.Nevertheless,existing optimizers often focus on locating non-zero variable posi-tions to optimize the binary variables Mask.However,approxi-mating the sparse distribution of real Pareto optimal solutions does not necessarily mean that the objective function is optimized.In data mining,it is common to mine frequent itemsets appear-ing together in a dataset to reveal the correlation between data.Inspired by this,we propose a novel two-layer encoding learning swarm optimizer based on frequent itemsets(TELSO)to address these SLMOPs.TELSO mined the frequent terms of multiple particles with better target values to find mask combinations that can obtain better objective values for fast convergence.Experi-mental results on five real-world problems and eight benchmark sets demonstrate that TELSO outperforms existing state-of-the-art sparse large-scale multi-objective evolutionary algorithms(SLMOEAs)in terms of performance and convergence speed.

A Fast Clustering Based Evolutionary Algorithm for Super-Large-Scale Sparse Multi-Objective Optimization

During the last three decades,evolutionary algorithms(EAs)have shown superiority in solving complex optimization problems,especially those with multiple objectives and non-differentiable landscapes.However,due to the stochastic search strategies,the performance of most EAs deteriorates drastically when handling a large number of decision variables.To tackle the curse of dimensionality,this work proposes an efficient EA for solving super-large-scale multi-objective optimization problems with sparse optimal solutions.The proposed algorithm estimates the sparse distribution of optimal solutions by optimizing a binary vector for each solution,and provides a fast clustering method to highly reduce the dimensionality of the search space.More importantly,all the operations related to the decision variables only contain several matrix calculations,which can be directly accelerated by GPUs.While existing EAs are capable of handling fewer than 10000 real variables,the proposed algorithm is verified to be effective in handling 1000000 real variables.Furthermore,since the proposed algorithm handles the large number of variables via accelerated matrix calculations,its runtime can be reduced to less than 10%of the runtime of existing EAs.

Constrained Multi-Objective Optimization With Deep Reinforcement Learning Assisted Operator Selection

Solving constrained multi-objective optimization problems with evolutionary algorithms has attracted considerable attention.Various constrained multi-objective optimization evolutionary algorithms(CMOEAs)have been developed with the use of different algorithmic strategies,evolutionary operators,and constraint-handling techniques.The performance of CMOEAs may be heavily dependent on the operators used,however,it is usually difficult to select suitable operators for the problem at hand.Hence,improving operator selection is promising and necessary for CMOEAs.This work proposes an online operator selection framework assisted by Deep Reinforcement Learning.The dynamics of the population,including convergence,diversity,and feasibility,are regarded as the state;the candidate operators are considered as actions;and the improvement of the population state is treated as the reward.By using a Q-network to learn a policy to estimate the Q-values of all actions,the proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state and thereby improve the algorithmic performance.The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems.The experimental results reveal that the proposed Deep Reinforcement Learning-assisted operator selection significantly improves the performance of these CMOEAs and the resulting algorithm obtains better versatility compared to nine state-of-the-art CMOEAs.

Even Search in a Promising Region for Constrained Multi-Objective Optimization

In recent years, a large number of approaches to constrained multi-objective optimization problems(CMOPs) have been proposed, focusing on developing tweaked strategies and techniques for handling constraints. However, an overly finetuned strategy or technique might overfit some problem types,resulting in a lack of versatility. In this article, we propose a generic search strategy that performs an even search in a promising region. The promising region, determined by obtained feasible non-dominated solutions, possesses two general properties.First, the constrained Pareto front(CPF) is included in the promising region. Second, as the number of feasible solutions increases or the convergence performance(i.e., approximation to the CPF) of these solutions improves, the promising region shrinks. Then we develop a new strategy named even search,which utilizes the non-dominated solutions to accelerate convergence and escape from local optima, and the feasible solutions under a constraint relaxation condition to exploit and detect feasible regions. Finally, a diversity measure is adopted to make sure that the individuals in the population evenly cover the valuable areas in the promising region. Experimental results on 45 instances from four benchmark test suites and 14 real-world CMOPs have demonstrated that searching evenly in the promising region can achieve competitive performance and excellent versatility compared to 11 most state-of-the-art methods tailored for CMOPs.

Multi-Objective Optimization Algorithm for Grouping Decision Variables Based on Extreme Point Pareto Frontier

The existing algorithms for solving multi-objective optimization problems fall into three main categories:Decomposition-based,dominance-based,and indicator-based.Traditional multi-objective optimization problemsmainly focus on objectives,treating decision variables as a total variable to solve the problem without consideringthe critical role of decision variables in objective optimization.As seen,a variety of decision variable groupingalgorithms have been proposed.However,these algorithms are relatively broad for the changes of most decisionvariables in the evolution process and are time-consuming in the process of finding the Pareto frontier.To solvethese problems,a multi-objective optimization algorithm for grouping decision variables based on extreme pointPareto frontier(MOEA-DV/EPF)is proposed.This algorithm adopts a preprocessing rule to solve the Paretooptimal solution set of extreme points generated by simultaneous evolution in various target directions,obtainsthe basic Pareto front surface to determine the convergence effect,and analyzes the convergence and distributioneffects of decision variables.In the later stages of algorithm optimization,different mutation strategies are adoptedaccording to the nature of the decision variables to speed up the rate of evolution to obtain excellent individuals,thusenhancing the performance of the algorithm.Evaluation validation of the test functions shows that this algorithmcan solve the multi-objective optimization problem more efficiently.

Evolutionary Multitasking With Global and Local Auxiliary Tasks for Constrained Multi-Objective Optimization

Constrained multi-objective optimization problems(CMOPs) include the optimization of objective functions and the satisfaction of constraint conditions, which challenge the solvers.To solve CMOPs, constrained multi-objective evolutionary algorithms(CMOEAs) have been developed. However, most of them tend to converge into local areas due to the loss of diversity. Evolutionary multitasking(EMT) is new model of solving complex optimization problems, through the knowledge transfer between the source task and other related tasks. Inspired by EMT, this paper develops a new EMT-based CMOEA to solve CMOPs, in which the main task, a global auxiliary task, and a local auxiliary task are created and optimized by one specific population respectively. The main task focuses on finding the feasible Pareto front(PF), and global and local auxiliary tasks are used to respectively enhance global and local diversity. Moreover, the global auxiliary task is used to implement the global search by ignoring constraints, so as to help the population of the main task pass through infeasible obstacles. The local auxiliary task is used to provide local diversity around the population of the main task, so as to exploit promising regions. Through the knowledge transfer among the three tasks, the search ability of the population of the main task will be significantly improved. Compared with other state-of-the-art CMOEAs, the experimental results on three benchmark test suites demonstrate the superior or competitive performance of the proposed CMOEA.

An Opposition-Based Learning-Based Search Mechanism for Flying Foxes Optimization Algorithm

The flying foxes optimization(FFO)algorithm,as a newly introduced metaheuristic algorithm,is inspired by the survival tactics of flying foxes in heat wave environments.FFO preferentially selects the best-performing individuals.This tendency will cause the newly generated solution to remain closely tied to the candidate optimal in the search area.To address this issue,the paper introduces an opposition-based learning-based search mechanism for FFO algorithm(IFFO).Firstly,this paper introduces niching techniques to improve the survival list method,which not only focuses on the adaptability of individuals but also considers the population’s crowding degree to enhance the global search capability.Secondly,an initialization strategy of opposition-based learning is used to perturb the initial population and elevate its quality.Finally,to verify the superiority of the improved search mechanism,IFFO,FFO and the cutting-edge metaheuristic algorithms are compared and analyzed using a set of test functions.The results prove that compared with other algorithms,IFFO is characterized by its rapid convergence,precise results and robust stability.

Evolutionary-assisted reinforcement learning for reservoir real-time production optimization under uncertainty

Production optimization has gained increasing attention from the smart oilfield community because it can increase economic benefits and oil recovery substantially.While existing methods could produce high-optimality results,they cannot be applied to real-time optimization for large-scale reservoirs due to high computational demands.In addition,most methods generally assume that the reservoir model is deterministic and ignore the uncertainty of the subsurface environment,making the obtained scheme unreliable for practical deployment.In this work,an efficient and robust method,namely evolutionaryassisted reinforcement learning(EARL),is proposed to achieve real-time production optimization under uncertainty.Specifically,the production optimization problem is modeled as a Markov decision process in which a reinforcement learning agent interacts with the reservoir simulator to train a control policy that maximizes the specified goals.To deal with the problems of brittle convergence properties and lack of efficient exploration strategies of reinforcement learning approaches,a population-based evolutionary algorithm is introduced to assist the training of agents,which provides diverse exploration experiences and promotes stability and robustness due to its inherent redundancy.Compared with prior methods that only optimize a solution for a particular scenario,the proposed approach trains a policy that can adapt to uncertain environments and make real-time decisions to cope with unknown changes.The trained policy,represented by a deep convolutional neural network,can adaptively adjust the well controls based on different reservoir states.Simulation results on two reservoir models show that the proposed approach not only outperforms the RL and EA methods in terms of optimization efficiency but also has strong robustness and real-time decision capacity.

A Reference Vector-Assisted Many-Objective Optimization Algorithm with Adaptive Niche Dominance Relation

It is still a huge challenge for traditional Pareto-dominatedmany-objective optimization algorithms to solve manyobjective optimization problems because these algorithms hardly maintain the balance between convergence and diversity and can only find a group of solutions focused on a small area on the Pareto front,resulting in poor performance of those algorithms.For this reason,we propose a reference vector-assisted algorithmwith an adaptive niche dominance relation,for short MaOEA-AR.The new dominance relation forms a niche based on the angle between candidate solutions.By comparing these solutions,the solutionwith the best convergence is found to be the non-dominated solution to improve the selection pressure.In reproduction,a mutation strategy of k-bit crossover and hybrid mutation is used to generate high-quality offspring.On 23 test problems with up to 15-objective,we compared the proposed algorithm with five state-of-the-art algorithms.The experimental results verified that the proposed algorithm is competitive.

Evolutionary Multitask Optimization in Real-World Applications: A Survey

Because of its strong ability to solve problems,evolutionary multitask optimization(EMTO)algorithms have been widely studied recently.Evolutionary algorithms have the advantage of fast searching for the optimal solution,but it is easy to fall into local optimum and difficult to generalize.Combining evolutionary multitask algorithms with evolutionary optimization algorithms can be an effective method for solving these problems.Through the implicit parallelism of tasks themselves and the knowledge transfer between tasks,more promising individual algorithms can be generated in the evolution process,which can jump out of the local optimum.How to better combine the two has also been studied more and more.This paper explores the existing evolutionary multitasking theory and improvement scheme in detail.Then,it summarizes the application of EMTO in different scenarios.Finally,according to the existing research,the future research trends and potential exploration directions are revealed.

A Modified Bi-Directional Evolutionary Structural Optimization Procedure with Variable Evolutionary Volume Ratio Applied to Multi-Objective Topology Optimization Problem

Natural frequency and dynamic stiffness under transient loading are two key performances for structural design related to automotive,aviation and construction industries.This article aims to tackle the multi-objective topological optimization problem considering dynamic stiffness and natural frequency using modified version of bi-directional evolutionary structural optimization(BESO).The conventional BESO is provided with constant evolutionary volume ratio(EVR),whereas low EVR greatly retards the optimization process and high EVR improperly removes the efficient elements.To address the issue,the modified BESO with variable EVR is introduced.To compromise the natural frequency and the dynamic stiffness,a weighting scheme of sensitivity numbers is employed to form the Pareto solution space.Several numerical examples demonstrate that the optimal solutions obtained from the modified BESO method have good agreement with those from the classic BESO method.Most importantly,the dynamic removal strategy with the variable EVR sharply springs up the optimization process.Therefore,it is concluded that the modified BESO method with variable EVR can solve structural design problems using multi-objective optimization.

A Smooth Bidirectional Evolutionary Structural Optimization of Vibrational Structures for Natural Frequency and Dynamic Compliance

A smooth bidirectional evolutionary structural optimization(SBESO),as a bidirectional version of SESO is proposed to solve the topological optimization of vibrating continuum structures for natural frequencies and dynamic compliance under the transient load.A weighted function is introduced to regulate the mass and stiffness matrix of an element,which has the inefficient element gradually removed from the design domain as if it were undergoing damage.Aiming at maximizing the natural frequency of a structure,the frequency optimization formulation is proposed using the SBESO technique.The effects of various weight functions including constant,linear and sine functions on structural optimization are compared.With the equivalent static load(ESL)method,the dynamic stiffness optimization of a structure is formulated by the SBESO technique.Numerical examples show that compared with the classic BESO method,the SBESO method can efficiently suppress the excessive element deletion by adjusting the element deletion rate and weight function.It is also found that the proposed SBESO technique can obtain an efficient configuration and smooth boundary and demonstrate the advantages over the classic BESO technique.

Multi-objective integrated optimization based on evolutionary strategy with a dynamic weighting schedule

The evolutionary strategy with a dynamic weighting schedule is proposed to find all the compromised solutions of the multi-objective integrated structure and control optimization problem, where the optimal system performance and control cost are defined by H2 or H∞ norms. During this optimization process, the weights are varying with the increasing generation instead of fixed values. The proposed strategy together with the linear matrix inequality （LMI） or the Riccati controller design method can find a series of uniformly distributed nondominated solutions in a single run. Therefore, this method can greatly reduce the computation intensity of the integrated optimization problem compared with the weight-based single objective genetic algorithm. Active automotive suspension is adopted as an example to illustrate the effectiveness of the proposed method.

A Chaotic Local Search-Based Particle Swarm Optimizer for Large-Scale Complex Wind Farm Layout Optimization

Wind energy has been widely applied in power generation to alleviate climate problems.The wind turbine layout of a wind farm is a primary factor of impacting power conversion efficiency due to the wake effect that reduces the power outputs of wind turbines located in downstream.Wind farm layout optimization(WFLO)aims to reduce the wake effect for maximizing the power outputs of the wind farm.Nevertheless,the wake effect among wind turbines increases significantly as the number of wind turbines increases in the wind farm,which severely affect power conversion efficiency.Conventional heuristic algorithms suffer from issues of low solution quality and local optimum for large-scale WFLO under complex wind scenarios.Thus,a chaotic local search-based genetic learning particle swarm optimizer(CGPSO)is proposed to optimize large-scale WFLO problems.CGPSO is tested on four larger-scale wind farms under four complex wind scenarios and compares with eight state-of-the-art algorithms.The experiment results indicate that CGPSO significantly outperforms its competitors in terms of performance,stability,and robustness.To be specific,a success and failure memories-based selection is proposed to choose a chaotic map for chaotic search local.It improves the solution quality.The parameter and search pattern of chaotic local search are also analyzed for WFLO problems.

Al-Biruni Earth Radius(BER)Metaheuristic Search Optimization Algorithm

Metaheuristic optimization algorithms present an effective method for solving several optimization problems from various types of applications and fields.Several metaheuristics and evolutionary optimization algorithms have been emerged recently in the literature and gained widespread attention,such as particle swarm optimization(PSO),whale optimization algorithm(WOA),grey wolf optimization algorithm(GWO),genetic algorithm(GA),and gravitational search algorithm(GSA).According to the literature,no one metaheuristic optimization algorithm can handle all present optimization problems.Hence novel optimization methodologies are still needed.The Al-Biruni earth radius(BER)search optimization algorithm is proposed in this paper.The proposed algorithm was motivated by the behavior of swarm members in achieving their global goals.The search space around local solutions to be explored is determined by Al-Biruni earth radius calculation method.A comparative analysis with existing state-of-the-art optimization algorithms corroborated the findings of BER’s validation and testing against seven mathematical optimization problems.The results show that BER can both explore and avoid local optima.BER has also been tested on an engineering design optimization problem.The results reveal that,in terms of performance and capability,BER outperforms the performance of state-of-the-art metaheuristic optimization algorithms.

A New Evolutionary Algorithm for Function Optimization

A new algorithm based on genetic algorithm(GA) is developed for solving function optimization problems with inequality constraints. This algorithm has been used to a series of standard test problems and exhibited good performance. The computation results show that its generality, precision, robustness, simplicity and performance are all satisfactory.