Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the l...Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the lower bound of the trust-region subproblem by considering the negative gradient direction. In this article, we give an alternate way to estimate the same lower bound of the trust-region subproblem.展开更多
In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorith...In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorithm under reasonable assumptions.展开更多
In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of comput...In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of computing trial directions by this method combining with the line search technique. Close to the solution set this method is locally Q-superlinearly convergent under an error bound assumption which is much weaker than the standard nonsingularity condition.展开更多
In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. ...In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. Numerical results are reported.展开更多
The speeding-up and slowing-down(SUSD)direction is a novel direction,which is proved to converge to the gradient descent direction under some conditions.The authors propose the derivative-free optimization algorithm S...The speeding-up and slowing-down(SUSD)direction is a novel direction,which is proved to converge to the gradient descent direction under some conditions.The authors propose the derivative-free optimization algorithm SUSD-TR,which combines the SUSD direction based on the covariance matrix of interpolation points and the solution of the trust-region subproblem of the interpolation model function at the current iteration step.They analyze the optimization dynamics and convergence of the algorithm SUSD-TR.Details of the trial step and structure step are given.Numerical results show their algorithm’s efficiency,and the comparison indicates that SUSD-TR greatly improves the method’s performance based on the method that only goes along the SUSD direction.Their algorithm is competitive with state-of-the-art mathematical derivative-free optimization algorithms.展开更多
We propose a trust-region type method for a class of nonsmooth nonconvex optimization problems where the objective function is a summation of a(probably nonconvex)smooth function and a(probably nonsmooth)convex functi...We propose a trust-region type method for a class of nonsmooth nonconvex optimization problems where the objective function is a summation of a(probably nonconvex)smooth function and a(probably nonsmooth)convex function.The model function of our trust-region subproblem is always quadratic and the linear term of the model is generated using abstract descent directions.Therefore,the trust-region subproblems can be easily constructed as well as efficiently solved by cheap and standard methods.When the accuracy of the model function at the solution of the subproblem is not sufficient,we add a safeguard on the stepsizes for improving the accuracy.For a class of functions that can be“truncated”,an additional truncation step is defined and a stepsize modification strategy is designed.The overall scheme converges globally and we establish fast local convergence under suitable assumptions.In particular,using a connection with a smooth Riemannian trust-region method,we prove local quadratic convergence for partly smooth functions under a strict complementary condition.Preliminary numerical results on a family of Ei-optimization problems are reported and demonstrate the eficiency of our approach.展开更多
We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends li...We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends linearly on the gradient used to define the latest model.The complexity results of the STRME method in nonconvex,convex and strongly convex settings are presented,which match those of the existing algorithms based on probabilistic properties.In addition,several numerical experiments are carried out to reveal the benefits of the proposed methods compared to the existing stochastic trust-region methods and other relevant stochastic gradient methods.展开更多
Because of its vital role of the trust-region subproblem (TRS) in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-s...Because of its vital role of the trust-region subproblem (TRS) in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-scale sparse TRS. The truncated Lanczos approach proposed by N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint [SIAM J. Optim., 1999, 9: 504-525] is a natural extension of the classical Lanczos method for the symmetric linear system and eigenvalue problem and, indeed follows the classical Rayleigh-Ritz procedure for eigenvalue computations. It consists of 1) projecting the original TRS to the Krylov subspa^es to yield smaller size TRS's and then 2) solving the resulted TRS's to get the approximates of the original TRS. This paper presents a posterior error bounds for both the global optimal value and the optimal solution between the original TRS and their projected counterparts. Our error bounds mainly rely on the factors from the Lanczos process as well as the data of the original TRS and, could be helpful in designing certain stopping criteria for the truncated Lanczos approach.展开更多
In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different wit...In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different with the traditional algorithms that usually require to solve two quadratic subproblems. Moreover, to avoid Maratos effect, the nonmonotone strategy is employed. The analysis shows that, under standard conditions, the algorithm has global and superlinear convergence. Preliminary numerical experiments are conducted to show the effiency of the new method.展开更多
This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wa...This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wang and Yuan(Numer.Math.104:241–269,2006)for the standard trust-region subproblem.Under suitable conditions,it is shown that the trial step obtained from the CDT subproblem is in the subspace spanned by all the gradient vectors of the objective function and of the constraints computed until the current iteration.Based on this observation,a subspace version of the Powell–Yuan trust-region algorithm is proposed for equality constrained optimization problems where the number of constraints is much lower than the number of variables. The convergence analysis is given and numerical results arealso reported.展开更多
A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new alg...A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new algorithm satisfies the first order necessary condition and there exists at least one limit point ofthe sequence which satisfies the second order necessary condition. Some preliminary numerical experiments are reported.展开更多
This paper presents a new trust-region algorithm for n-dimension nonlinear optimization subject to m nonlinear inequality constraints. Equivalent KKT conditions are derived, which is the basis for constructing the new...This paper presents a new trust-region algorithm for n-dimension nonlinear optimization subject to m nonlinear inequality constraints. Equivalent KKT conditions are derived, which is the basis for constructing the new algorithm. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps, local quadratic convergence theorem is proved for nondegenerate minimizer point. Numerical experiment is presented to show the effectiveness of our approach.展开更多
In this article,we consider the primal-dual path-following method and the trust-region updating strategy for the standard linear programming problem.For the rank-deficient problem with the small noisy data,we also giv...In this article,we consider the primal-dual path-following method and the trust-region updating strategy for the standard linear programming problem.For the rank-deficient problem with the small noisy data,we also give the preprocessing method based on the QR decomposition with column pivoting.Then,we prove the global convergence of the new method when the initial point is strictly primal-dual feasible.Finally,for some rankdeficient problems with or without the small noisy data from the NETLIB collection,we compare it with other two popular interior-point methods,i.e.the subroutine pathfollow.m and the built-in subroutine linprog.m of the MATLAB environment.Numerical results show that the new method is more robust than the other two methods for the rank-deficient problem with the small noise data.展开更多
文摘Trust-region methods are popular for nonlinear optimization problems. How to determine the predicted reduction of the trust-region subproblem is a key issue for trust-region methods. Powell gave an estimation of the lower bound of the trust-region subproblem by considering the negative gradient direction. In this article, we give an alternate way to estimate the same lower bound of the trust-region subproblem.
基金Supported by the National Natural Science Foundation of China (10231060), the Special Research Found of Doctoral Program of Higher Education of China(200d0319003 ), the Research Project of Xuzhou Institute of Technology( XKY200622).
基金Supported by CERG: CityU 101005 of the Government of Hong Kong SAR, Chinathe National Natural ScienceFoundation of China, the Specialized Research Fund of Doctoral Program of Higher Education of China (Grant No.20040319003)the Natural Science Fund of Jiangsu Province of China (Grant No. BK2006214)
文摘In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorithm under reasonable assumptions.
基金Supported by the National Natural Science Foundation of China (10871130)the Research Fund for the Doctoral Program of Higher Education of China (20093127110005)the Scientific Computing Key Laboratory of Shanghai Universities
文摘In this paper, a projected gradient trust region algorithm for solving nonlinear equality systems with convex constraints is considered. The global convergence results are developed in a very general setting of computing trial directions by this method combining with the line search technique. Close to the solution set this method is locally Q-superlinearly convergent under an error bound assumption which is much weaker than the standard nonsingularity condition.
文摘In this paper, we propose a nonmonotone adap-tive trust-region method for solving symmetric nonlinear equations problems. The convergent result of the presented method will be estab-lished under favorable conditions. Numerical results are reported.
基金supported by the National Natural Science Foundation of China(No.12288201)。
文摘The speeding-up and slowing-down(SUSD)direction is a novel direction,which is proved to converge to the gradient descent direction under some conditions.The authors propose the derivative-free optimization algorithm SUSD-TR,which combines the SUSD direction based on the covariance matrix of interpolation points and the solution of the trust-region subproblem of the interpolation model function at the current iteration step.They analyze the optimization dynamics and convergence of the algorithm SUSD-TR.Details of the trial step and structure step are given.Numerical results show their algorithm’s efficiency,and the comparison indicates that SUSD-TR greatly improves the method’s performance based on the method that only goes along the SUSD direction.Their algorithm is competitive with state-of-the-art mathematical derivative-free optimization algorithms.
基金partly supported by the Fundamental Research Fund-Shenzhen Research Institute for Big Data(SRIBD)Startup Fund JCYJ-AM20190601partly supported by the NSFC grant 11831002the Beijing Academy of Artificial Intelligence.
文摘We propose a trust-region type method for a class of nonsmooth nonconvex optimization problems where the objective function is a summation of a(probably nonconvex)smooth function and a(probably nonsmooth)convex function.The model function of our trust-region subproblem is always quadratic and the linear term of the model is generated using abstract descent directions.Therefore,the trust-region subproblems can be easily constructed as well as efficiently solved by cheap and standard methods.When the accuracy of the model function at the solution of the subproblem is not sufficient,we add a safeguard on the stepsizes for improving the accuracy.For a class of functions that can be“truncated”,an additional truncation step is defined and a stepsize modification strategy is designed.The overall scheme converges globally and we establish fast local convergence under suitable assumptions.In particular,using a connection with a smooth Riemannian trust-region method,we prove local quadratic convergence for partly smooth functions under a strict complementary condition.Preliminary numerical results on a family of Ei-optimization problems are reported and demonstrate the eficiency of our approach.
基金This research is partially supported by the National Natural Science Foundation of China 11331012 and 11688101.
文摘We present a stochastic trust-region model-based framework in which its radius is related to the probabilistic models.Especially,we propose a specific algorithm termed STRME,in which the trust-region radius depends linearly on the gradient used to define the latest model.The complexity results of the STRME method in nonconvex,convex and strongly convex settings are presented,which match those of the existing algorithms based on probabilistic properties.In addition,several numerical experiments are carried out to reveal the benefits of the proposed methods compared to the existing stochastic trust-region methods and other relevant stochastic gradient methods.
基金The authors would like to thank the anonymous referees for their careful reading and comments. This work of the first author was supported in part by the National Natural Science Foundation of China (Grant Nos. 11671246, 91730303, 11371102) and the work of the second author was supported in part by the National Natural Science Foundation of China (Grant Nos. 91730304, 11371102, 91330201).
文摘Because of its vital role of the trust-region subproblem (TRS) in various applications, for example, in optimization and in ill-posed problems, there are several factorization-free algorithms for solving the large-scale sparse TRS. The truncated Lanczos approach proposed by N. I. M. Gould, S. Lucidi, M. Roma, and P. L. Toint [SIAM J. Optim., 1999, 9: 504-525] is a natural extension of the classical Lanczos method for the symmetric linear system and eigenvalue problem and, indeed follows the classical Rayleigh-Ritz procedure for eigenvalue computations. It consists of 1) projecting the original TRS to the Krylov subspa^es to yield smaller size TRS's and then 2) solving the resulted TRS's to get the approximates of the original TRS. This paper presents a posterior error bounds for both the global optimal value and the optimal solution between the original TRS and their projected counterparts. Our error bounds mainly rely on the factors from the Lanczos process as well as the data of the original TRS and, could be helpful in designing certain stopping criteria for the truncated Lanczos approach.
文摘In this paper, a new trust region algorithm for minimax optimization problems is proposed, which solves only one quadratic subproblem based on a new approximation model at each iteration. The approach is different with the traditional algorithms that usually require to solve two quadratic subproblems. Moreover, to avoid Maratos effect, the nonmonotone strategy is employed. The analysis shows that, under standard conditions, the algorithm has global and superlinear convergence. Preliminary numerical experiments are conducted to show the effiency of the new method.
基金G.N.Grapiglia was supported by Coordination for the Improvement of Higher Education Personnel(CAPES),Brazil(Grant PGCI No.12347/12-4).J.Yuan was partially supported by Coordination for the Improvement of Higher Education Personnel(CAPES)and by the National Council for Scientific and Technological Development(CNPq),Brazil.Y.-x.Yuan was partially supported by Natural Science Foundation of China,China(Grant No.11331012)This work was carried out while the first author was visiting Institute of Computational Mathematics and Scientific/Engineering Computing of the Chinese Academy of Sciences.He would like to thank Professor Ya-xiang Yuan,Professor Yu-hong Dai,Dr.Xin Liu and Dr.Ya-feng Liu for their warm hospitality.The authors also are grateful to Dr.Wei Leng for his help in installing and configuring the CUTEr.Finally,the authors would like to thank the two referees for their helpful comments.
文摘This paper studied subspace properties of the Celis–Dennis–Tapia(CDT)subproblem that arises in some trust-region algorithms for equality constrained optimization.The analysis is an extension of that presented by Wang and Yuan(Numer.Math.104:241–269,2006)for the standard trust-region subproblem.Under suitable conditions,it is shown that the trial step obtained from the CDT subproblem is in the subspace spanned by all the gradient vectors of the objective function and of the constraints computed until the current iteration.Based on this observation,a subspace version of the Powell–Yuan trust-region algorithm is proposed for equality constrained optimization problems where the number of constraints is much lower than the number of variables. The convergence analysis is given and numerical results arealso reported.
基金This work was supported by the National Natural Science Foundation of China(Crant No.39830070).
文摘A new trust-region and affine scaling algorithm for linearly constrained optimization is presentedin this paper. Under no nondegenerate assumption, we prove that any limit point of the sequence generatedby the new algorithm satisfies the first order necessary condition and there exists at least one limit point ofthe sequence which satisfies the second order necessary condition. Some preliminary numerical experiments are reported.
基金the Scientific Research Foundation of Hunan Provincial Education Department, No. 02B021, and the National Natural Science Foundation of China, No. 10171008.
文摘This paper presents a new trust-region algorithm for n-dimension nonlinear optimization subject to m nonlinear inequality constraints. Equivalent KKT conditions are derived, which is the basis for constructing the new algorithm. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps, local quadratic convergence theorem is proved for nondegenerate minimizer point. Numerical experiment is presented to show the effectiveness of our approach.
基金This work was supported in part by Grant 61876199 from National Natural Science Foundation of China,Grant YBWL2011085 from Huawei Technologies Co.,Ltd.,and Grant YJCB2011003HIInnovation Research Program of Huawei Technologies Co.,Ltd..The first author is grateful to professor Li-Zhi Liao for introducing him the interiorpoint methods when he visited Hong Kong Baptist University in July,2012.
文摘In this article,we consider the primal-dual path-following method and the trust-region updating strategy for the standard linear programming problem.For the rank-deficient problem with the small noisy data,we also give the preprocessing method based on the QR decomposition with column pivoting.Then,we prove the global convergence of the new method when the initial point is strictly primal-dual feasible.Finally,for some rankdeficient problems with or without the small noisy data from the NETLIB collection,we compare it with other two popular interior-point methods,i.e.the subroutine pathfollow.m and the built-in subroutine linprog.m of the MATLAB environment.Numerical results show that the new method is more robust than the other two methods for the rank-deficient problem with the small noise data.