Subspace Minimization Conjugate Gradient Method Based on Cubic Regularization Model for Unconstrained Optimization

Many methods have been put forward to solve unconstrained optimization problems,among which conjugate gradient method(CG)is very important.With the increasing emergence of large⁃scale problems,the subspace technology has become particularly important and widely used in the field of optimization.In this study,a new CG method was put forward,which combined subspace technology and a cubic regularization model.Besides,a special scaled norm in a cubic regularization model was analyzed.Under certain conditions,some significant characteristics of the search direction were given and the convergence of the algorithm was built.Numerical comparisons show that for the 145 test functions under the CUTEr library,the proposed method is better than two classical CG methods and two new subspaces conjugate gradient methods.

Global Convergence of Curve Search Methods for Unconstrained Optimization

In this paper we propose a new family of curve search methods for unconstrained optimization problems, which are based on searching a new iterate along a curve through the current iterate at each iteration, while line search methods are based on finding a new iterate on a line starting from the current iterate at each iteration. The global convergence and linear convergence rate of these curve search methods are investigated under some mild conditions. Numerical results show that some curve search methods are stable and effective in solving some large scale minimization problems.

NON-QUASI-NEWTON UPDATES FOR UNCONSTRAINED OPTIMIZATION

In this report we present some new numerical methods for unconstrained op-timization. These methods apply update formulae that do not satisfy the quasi-Newton equation. We derive these new formulae by considering different techniques of approkimating the objective function. Theoretical analyses are given to show the advantages of using non-quasi-Newton updates. Under mild conditions we prove that our new update formulae preserve global convergence properties. Numerical results are also presented.

A New Restarting Adaptive Trust-Region Method for Unconstrained Optimization

In this paper,we present a new adaptive trust-region method for solving nonlinear unconstrained optimization problems.More precisely,a trust-region radius based on a nonmonotone technique uses an approximation of Hessian which is adaptively chosen.We produce a suitable trust-region radius;preserve the global convergence under classical assumptions to the first-order critical points;improve the practical performance of the new algorithm compared to other exiting variants.Moreover,the quadratic convergence rate is established under suitable conditions.Computational results on the CUTEst test collection of unconstrained problems are presented to show the effectiveness of the proposed algorithm compared with some exiting methods.

A randomized nonmonotone adaptive trust region method based on the simulated annealing strategy for unconstrained optimization

Purpose–The purpose of this paper is to employ stochastic techniques to increase efficiency of the classical algorithms for solving nonlinear optimization problems.Design/methodology/approach–The well-known simulated annealing strategy is employed to search successive neighborhoods of the classical trust region(TR)algorithm.Findings–An adaptive formula for computing the TR radius is suggested based on an eigenvalue analysis conducted on the memoryless Broyden-Fletcher-Goldfarb-Shanno updating formula.Also,a(heuristic)randomized adaptive TR algorithm is developed for solving unconstrained optimization problems.Results of computational experiments on a set of CUTEr test problems show that the proposed randomization scheme can enhance efficiency of the TR methods.Practical implications–The algorithm can be effectively used for solving the optimization problems which appear in engineering,economics,management,industry and other areas.Originality/value–The proposed randomization scheme improves computational costs of the classical TR algorithm.Especially,the suggested algorithm avoids resolving the TR subproblems for many times.

A Regularized Newton Method with Correction for Unconstrained Convex Optimization

In this paper, we present a regularized Newton method (M-RNM) with correction for minimizing a convex function whose Hessian matrices may be singular. At every iteration, not only a RNM step is computed but also two correction steps are computed. We show that if the objective function is LC2, then the method posses globally convergent. Numerical results show that the new algorithm performs very well.

Chaotic Aquila Optimization Algorithm for Solving Phase Equilibrium Problems and Parameter Estimation of Semi-empirical Models

This research study aims to enhance the optimization performance of a newly emerged Aquila Optimization algorithm by incorporating chaotic sequences rather than using uniformly generated Gaussian random numbers.This work employs 25 different chaotic maps under the framework of Aquila Optimizer.It considers the ten best chaotic variants for performance evaluation on multidimensional test functions composed of unimodal and multimodal problems,which have yet to be studied in past literature works.It was found that Ikeda chaotic map enhanced Aquila Optimization algorithm yields the best predictions and becomes the leading method in most of the cases.To test the effectivity of this chaotic variant on real-world optimization problems,it is employed on two constrained engineering design problems,and its effectiveness has been verified.Finally,phase equilibrium and semi-empirical parameter estimation problems have been solved by the proposed method,and respective solutions have been compared with those obtained from state-of-art optimizers.It is observed that CH01 can successfully cope with the restrictive nonlinearities and nonconvexities of parameter estimation and phase equilibrium problems,showing the capabilities of yielding minimum prediction error values of no more than 0.05 compared to the remaining algorithms utilized in the performance benchmarking process.

A New Nonmonotone Adaptive Trust Region Method

The trust region method plays an important role in solving optimization problems. In this paper, we propose a new nonmonotone adaptive trust region method for solving unconstrained optimization problems. Actually, we combine a popular nonmonotone technique with an adaptive trust region algorithm. The new ratio to adjusting the next trust region radius is different from the ratio in the traditional trust region methods. Under some appropriate conditions, we show that the new algorithm has good global convergence and superlinear convergence.

AN OPTIMAL SELF-SCALING STRATEGY TO THE MODIFIED SYMMETRIC RANK ONE UPDATING

In the paper, the optimal self-scaling strategy to the modified symmetric rank one (HSR1) update, which satisfies the modified quasi-Newton equation, is derived to improve the condition number of the updates. The scaling factors are derived from minimizing the estimate of upper bounds on the condition number of the updating matrix. Theoretical analysis, and numerical experiments and comparisons show that introducing the optimal scaling factor into the modified symmetric rank one update preserves the positive definiteness of updates, and greatly improves the stability and numerical performance of the modified symmetric rank one algorithm.

A Descent Gradient Method and Its Global Convergence

Y Liu and C Storey(1992)proposed the famous LS conjugate gradient method which has good numerical results.However,the LS method has very weak convergence under the Wolfe-type line search.In this paper,we give a new descent gradient method based on the LS method.It can guarantee the sufficient descent property at each iteration and the global convergence under the strong Wolfe line search.Finally,we also present extensive preliminary numerical experiments to show the efficiency of the proposed method by comparing with the famous PRP^+method.

Improved Dwarf Mongoose Optimization for Constrained Engineering Design Problems

This paper proposes a modified version of the Dwarf Mongoose Optimization Algorithm (IDMO) for constrained engineering design problems. This optimization technique modifies the base algorithm (DMO) in three simple but effective ways. First, the alpha selection in IDMO differs from the DMO, where evaluating the probability value of each fitness is just a computational overhead and contributes nothing to the quality of the alpha or other group members. The fittest dwarf mongoose is selected as the alpha, and a new operator ω is introduced, which controls the alpha movement, thereby enhancing the exploration ability and exploitability of the IDMO. Second, the scout group movements are modified by randomization to introduce diversity in the search process and explore unvisited areas. Finally, the babysitter's exchange criterium is modified such that once the criterium is met, the babysitters that are exchanged interact with the dwarf mongoose exchanging them to gain information about food sources and sleeping mounds, which could result in better-fitted mongooses instead of initializing them afresh as done in DMO, then the counter is reset to zero. The proposed IDMO was used to solve the classical and CEC 2020 benchmark functions and 12 continuous/discrete engineering optimization problems. The performance of the IDMO, using different performance metrics and statistical analysis, is compared with the DMO and eight other existing algorithms. In most cases, the results show that solutions achieved by the IDMO are better than those obtained by the existing algorithms.

Arc-search in numerical optimization

Determining the search direction and the search step are the two main steps of the nonlinear optimization algorithm,in which the derivatives of the objective and constraint functions are used to determine the search direction,the one-dimensional search and the trust domain methods are used to determine the step length along the search direction.One dimensional line search has been widely discussed in various textbooks and references.However,there is a lessknown techniquearc-search method,which is relatively new and may generate more efficient algorithms in some cases.In this paper,we will survey this technique,discuss its applications in different optimization problems,and explain its potential improvements over traditional line search method.

A new simple model trust-region method with generalized Barzilai-Borwein parameter for large-scale optimization

In this paper, a new trust region method with simple model for solving large-scale unconstrained nonlinear optimization is proposed. By employing the generalized weak quasi-Newton equations, we derive several schemes to construct variants of scalar matrices as the Hessian approximation used in the trust region subproblem. Under some reasonable conditions, global convergence of the proposed algorithm is established in the trust region framework. The numerical experiments on solving the test problems with dimensions from 50 to 20,000 in the CUTEr library are reported to show efficiency of the algorithm.

A new parameter setting-based modified differential evolution for function optimization

The population-based efficient iterative evolutionary algorithm(EA)is differential evolution(DE).It has fewer control parameters but is useful when dealing with complex problems of optimization in the real world.A great deal of progress has already been made and implemented in various fields of engineering and science.Nevertheless,DE is prone to the setting of control parameters in its performance evaluation.Therefore,the appropriate adjustment of the time-consuming control parameters is necessary to achieve optimal DE efficiency.This research proposes a new version of the DE algorithm control parameters and mutation operator.For the justifiability of the suggested method,several benchmark functions are taken from the literature.The test results are contrasted with other literary algorithms.

Shamanskii-Like Levenberg-Marquardt Method with a New Line Search for Systems of Nonlinear Equations

To save the calculations of Jacobian,a multi-step Levenberg-Marquardt method named Shamanskii-like LM method for systems of nonlinear equations was proposed by Fa.Its convergence properties have been proved by using a trust region technique under the local error bound condition.However,the authors wonder whether the similar convergence properties are still true with standard line searches since the direction may not be a descent direction.For this purpose,the authors present a new nonmonotone m-th order Armijo type line search to guarantee the global convergence.Under the same condition as trust region case,the convergence rate also has been shown to be m+1 by using this line search technique.Numerical experiments show the new algorithm can save much running time for the large scale problems,so it is efficient and promising.

Global Convergence of a Modified Spectral CD Conjugate Gradient Method

In this paper,we present a new nonlinear modified spectral CD conjugate gradient method for solving large scale unconstrained optimization problems.The direction generated by the method is a descent direction for the objective function,and this property depends neither on the line search rule,nor on the convexity of the objective function.Moreover,the modified method reduces to the standard CD method if line search is exact.Under some mild conditions,we prove that the modified method with line search is globally convergent even if the objective function is nonconvex.Preliminary numerical results show that the proposed method is very promising.

A New Analysis on the Barzilai-Borwein Gradient Method

Due to its simplicity and efficiency,the Barzilai and Borwein(BB)gradi-ent method has received various attentions in different fields.This paper presents a new analysis of the BB method for two-dimensional strictly convex quadratic func-tions.The analysis begins with the assumption that the gradient norms at the first two iterations are fixed.We show that there is a superlinear convergence step in at most three consecutive steps.Meanwhile,we provide a better convergence relation for the BB method.The influence of the starting point and the condition number to the convergence rate is comprehensively addressed.

LTSA-LE:A Local Tangent Space Alignment Label Enhancement Algorithm

According to smoothness assumption,local topological structure can be shared between feature and label manifolds.This study proposes a new algorithm based on Local Tangent Space Alignment(LTSA)to implement the label enhancement process.In general,we first establish a learning model for feature extraction in label space and use a feature extraction method of LTSA to guide the reconstruction of label manifolds.Then,we establish an unconstrained optimization model based on the optimal theory presented in this paper.The model is suitable for solving problems with a large number of sample points.Finally,the experiment results show that the algorithm can effectively improve the training speed and multilabel dataset prediction accuracy.

An asymptotically optimal public parking lot location algorithm based on intuitive reasoning

In order to solve the problems of road traffic congestion and the increasing parking time caused by the imbalance of parking lot supply and demand,this paper proposes an asymptotically optimal public parking lot location algorithm based on intuitive reasoning to optimize the parking lot location problem.Guided by the idea of intuitive reasoning,we use walking distance as indicator to measure the variability among location data and build a combinatorial optimization model aimed at guiding search decisions in the solution space of complex problems to find optimal solutions.First,Selective Attention Mechanism(SAM)is introduced to reduce the search space by adaptively focusing on the important information in the features.Then,Quantum Annealing(QA)algorithm with quantum tunneling effect is used to jump out of the local extremum in the search space with high probability and further approach the global optimal solution.Experiments on the parking lot location dataset in Luohu District,Shenzhen,show that the proposed method has improved the accuracy and running speed of the solution,and the asymptotic optimality of the algorithm and its effectiveness in solving the public parking lot location problem are verified.