Conjugate gradient method is one of successful methods for solving the unconstrained optimization problems. In this paper, absorbing the advantages of FR and CD methods, a hybrid conjugate gradient method is proposed....Conjugate gradient method is one of successful methods for solving the unconstrained optimization problems. In this paper, absorbing the advantages of FR and CD methods, a hybrid conjugate gradient method is proposed. Under the general Wolfe linear searches, the proposed method can generate the sufficient descent direction at each iterate,and its global convergence property also can be established. Some preliminary numerical results show that the proposed method is effective and stable for the given test problems.展开更多
This paper proposes a synchronous parallel block coordinate descent algorithm for minimizing a composite function,which consists of a smooth convex function plus a non-smooth but separable convex function.Due to the g...This paper proposes a synchronous parallel block coordinate descent algorithm for minimizing a composite function,which consists of a smooth convex function plus a non-smooth but separable convex function.Due to the generalization of the proposed method,some existing synchronous parallel algorithms can be considered as special cases.To tackle high dimensional problems,the authors further develop a randomized variant,which randomly update some blocks of coordinates at each round of computation.Both proposed parallel algorithms are proven to have sub-linear convergence rate under rather mild assumptions.The numerical experiments on solving the large scale regularized logistic regression with 1 norm penalty show that the implementation is quite efficient.The authors conclude with explanation on the observed experimental results and discussion on the potential improvements.展开更多
This paper presents a coordinate gradient descent approach for minimizing the sum of a smooth function and a nonseparable convex function.We find a search direction by solving a subproblem obtained by a second-order a...This paper presents a coordinate gradient descent approach for minimizing the sum of a smooth function and a nonseparable convex function.We find a search direction by solving a subproblem obtained by a second-order approximation of the smooth function and adding a separable convex function.Under a local Lipschitzian error bound assumption,we show that the algorithm possesses global and local linear convergence properties.We also give some numerical tests(including image recovery examples) to illustrate the efficiency of the proposed method.展开更多
This paper addresses the generalized linear complementarity problem （GLCP） over a polyhedral cone. To solve the problem, we first equivalently convert the problem into an affine variational inequalities problem over...This paper addresses the generalized linear complementarity problem （GLCP） over a polyhedral cone. To solve the problem, we first equivalently convert the problem into an affine variational inequalities problem over a closed polyhedral cone, and then propose a new type of method to solve the GLCP based on the error bound estimation. The global and R-linear convergence rate is established. The numerical experiments show the efficiency of the method.展开更多
In this paper, we present a modified projection method for the linear feasibility problems （LFP）. Compared with the existing methods, the new method adopts a surrogate technique to obtain new iteration instead of th...In this paper, we present a modified projection method for the linear feasibility problems （LFP）. Compared with the existing methods, the new method adopts a surrogate technique to obtain new iteration instead of the line search procedure with fixed stepsize. For the new method, we first show its global convergence under the condition that the solution set is nonempty, and then establish its linear convergence rate. Preliminary numerical experiments show that this method has good performance.展开更多
Consider the problem of minimizing the sum of two convex functions,one being smooth and the other non-smooth.In this paper,we introduce a general class of approximate proximal splitting(APS)methods for solving such mi...Consider the problem of minimizing the sum of two convex functions,one being smooth and the other non-smooth.In this paper,we introduce a general class of approximate proximal splitting(APS)methods for solving such minimization problems.Methods in the APS class include many well-known algorithms such as the proximal splitting method,the block coordinate descent method(BCD),and the approximate gradient projection methods for smooth convex optimization.We establish the linear convergence of APS methods under a local error bound assumption.Since the latter is known to hold for compressive sensing and sparse group LASSO problems,our analysis implies the linear convergence of the BCD method for these problems without strong convexity assumption.展开更多
文摘Conjugate gradient method is one of successful methods for solving the unconstrained optimization problems. In this paper, absorbing the advantages of FR and CD methods, a hybrid conjugate gradient method is proposed. Under the general Wolfe linear searches, the proposed method can generate the sufficient descent direction at each iterate,and its global convergence property also can be established. Some preliminary numerical results show that the proposed method is effective and stable for the given test problems.
基金supported by the National Key R&D Program of China under Grant No.2018YFC0830300。
文摘This paper proposes a synchronous parallel block coordinate descent algorithm for minimizing a composite function,which consists of a smooth convex function plus a non-smooth but separable convex function.Due to the generalization of the proposed method,some existing synchronous parallel algorithms can be considered as special cases.To tackle high dimensional problems,the authors further develop a randomized variant,which randomly update some blocks of coordinates at each round of computation.Both proposed parallel algorithms are proven to have sub-linear convergence rate under rather mild assumptions.The numerical experiments on solving the large scale regularized logistic regression with 1 norm penalty show that the implementation is quite efficient.The authors conclude with explanation on the observed experimental results and discussion on the potential improvements.
基金supported by NSFC Grant 10601043,NCETXMUSRF for ROCS,SEM+2 种基金supported by RGC 201508HKBU FRGssupported by the Hong Kong Research Grant Council
文摘This paper presents a coordinate gradient descent approach for minimizing the sum of a smooth function and a nonseparable convex function.We find a search direction by solving a subproblem obtained by a second-order approximation of the smooth function and adding a separable convex function.Under a local Lipschitzian error bound assumption,we show that the algorithm possesses global and local linear convergence properties.We also give some numerical tests(including image recovery examples) to illustrate the efficiency of the proposed method.
基金supported by National Natural Science Foundation of China (No. 10771120)
文摘This paper addresses the generalized linear complementarity problem （GLCP） over a polyhedral cone. To solve the problem, we first equivalently convert the problem into an affine variational inequalities problem over a closed polyhedral cone, and then propose a new type of method to solve the GLCP based on the error bound estimation. The global and R-linear convergence rate is established. The numerical experiments show the efficiency of the method.
基金supported by National Natural Science Foundation of China (No. 10771120)Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry
文摘In this paper, we present a modified projection method for the linear feasibility problems （LFP）. Compared with the existing methods, the new method adopts a surrogate technique to obtain new iteration instead of the line search procedure with fixed stepsize. For the new method, we first show its global convergence under the condition that the solution set is nonempty, and then establish its linear convergence rate. Preliminary numerical experiments show that this method has good performance.
文摘Consider the problem of minimizing the sum of two convex functions,one being smooth and the other non-smooth.In this paper,we introduce a general class of approximate proximal splitting(APS)methods for solving such minimization problems.Methods in the APS class include many well-known algorithms such as the proximal splitting method,the block coordinate descent method(BCD),and the approximate gradient projection methods for smooth convex optimization.We establish the linear convergence of APS methods under a local error bound assumption.Since the latter is known to hold for compressive sensing and sparse group LASSO problems,our analysis implies the linear convergence of the BCD method for these problems without strong convexity assumption.
