We propose a trust-region type method for a class of nonsmooth nonconvex optimization problems where the objective function is a summation of a(probably nonconvex)smooth function and a(probably nonsmooth)convex functi...We propose a trust-region type method for a class of nonsmooth nonconvex optimization problems where the objective function is a summation of a(probably nonconvex)smooth function and a(probably nonsmooth)convex function.The model function of our trust-region subproblem is always quadratic and the linear term of the model is generated using abstract descent directions.Therefore,the trust-region subproblems can be easily constructed as well as efficiently solved by cheap and standard methods.When the accuracy of the model function at the solution of the subproblem is not sufficient,we add a safeguard on the stepsizes for improving the accuracy.For a class of functions that can be“truncated”,an additional truncation step is defined and a stepsize modification strategy is designed.The overall scheme converges globally and we establish fast local convergence under suitable assumptions.In particular,using a connection with a smooth Riemannian trust-region method,we prove local quadratic convergence for partly smooth functions under a strict complementary condition.Preliminary numerical results on a family of Ei-optimization problems are reported and demonstrate the eficiency of our approach.展开更多
In this work,we present probabilistic local convergence results for a stochastic semismooth Newton method for a class of stochastic composite optimization problems involving the sum of smooth nonconvex and nonsmooth c...In this work,we present probabilistic local convergence results for a stochastic semismooth Newton method for a class of stochastic composite optimization problems involving the sum of smooth nonconvex and nonsmooth convex terms in the objective function.We assume that the gradient and Hessian information of the smooth part of the objective function can only be approximated and accessed via calling stochastic firstand second-order oracles.The approach combines stochastic semismooth Newton steps,stochastic proximal gradient steps and a globalization strategy based on growth conditions.We present tail bounds and matrix concentration inequalities for the stochastic oracles that can be utilized to control the approximation errors via appropriately adjusting or increasing the sampling rates.Under standard local assumptions,we prove that the proposed algorithm locally turns into a pure stochastic semismooth Newton method and converges r-linearly or r-superlinearly with high probability.展开更多
基金partly supported by the Fundamental Research Fund-Shenzhen Research Institute for Big Data(SRIBD)Startup Fund JCYJ-AM20190601partly supported by the NSFC grant 11831002the Beijing Academy of Artificial Intelligence.
文摘We propose a trust-region type method for a class of nonsmooth nonconvex optimization problems where the objective function is a summation of a(probably nonconvex)smooth function and a(probably nonsmooth)convex function.The model function of our trust-region subproblem is always quadratic and the linear term of the model is generated using abstract descent directions.Therefore,the trust-region subproblems can be easily constructed as well as efficiently solved by cheap and standard methods.When the accuracy of the model function at the solution of the subproblem is not sufficient,we add a safeguard on the stepsizes for improving the accuracy.For a class of functions that can be“truncated”,an additional truncation step is defined and a stepsize modification strategy is designed.The overall scheme converges globally and we establish fast local convergence under suitable assumptions.In particular,using a connection with a smooth Riemannian trust-region method,we prove local quadratic convergence for partly smooth functions under a strict complementary condition.Preliminary numerical results on a family of Ei-optimization problems are reported and demonstrate the eficiency of our approach.
基金supported by the Fundamental Research Fund—Shenzhen Research Institute for Big Data Startup Fund(Grant No.JCYJ-AM20190601)the Shenzhen Institute of Artificial Intelligence and Robotics for Society+2 种基金National Natural Science Foundation of China(Grant Nos.11831002 and 11871135)the Key-Area Research and Development Program of Guangdong Province(Grant No.2019B121204008)Beijing Academy of Artificial Intelligence。
文摘In this work,we present probabilistic local convergence results for a stochastic semismooth Newton method for a class of stochastic composite optimization problems involving the sum of smooth nonconvex and nonsmooth convex terms in the objective function.We assume that the gradient and Hessian information of the smooth part of the objective function can only be approximated and accessed via calling stochastic firstand second-order oracles.The approach combines stochastic semismooth Newton steps,stochastic proximal gradient steps and a globalization strategy based on growth conditions.We present tail bounds and matrix concentration inequalities for the stochastic oracles that can be utilized to control the approximation errors via appropriately adjusting or increasing the sampling rates.Under standard local assumptions,we prove that the proposed algorithm locally turns into a pure stochastic semismooth Newton method and converges r-linearly or r-superlinearly with high probability.