The era of big data in healthcare is here, and this era will significantly improve medicine and especially oncology. However, traditional machine learning algorithms need to be promoted to solve such large-scale real-...The era of big data in healthcare is here, and this era will significantly improve medicine and especially oncology. However, traditional machine learning algorithms need to be promoted to solve such large-scale real-world problems due to a large amount of data that needs to be analyzed and the difficulty in solving problems with nonconvex nonlinear settings. We aim to minimize the composite of a smooth nonlinear function and a block-separable nonconvex function on a large number of block variables with inequality constraints. We propose a novel parallel first-order optimization method, called asynchronous block coordinate descent with time perturbation (ATP), which adopts a time perturbation technique that escapes from saddle points and sub-optimal local points. The details of the proposed method are presented with analyses of convergence and iteration complexity properties. Experiments conducted on real-world machine learning problems validate the efficacy of our proposed method. The experimental results demonstrate that time perturbation enables ATP to escape from saddle points and sub-optimal points, providing a promising way to handle nonconvex optimization problems with inequality constraints employing asynchronous block coordinate descent. The asynchronous parallel implementation on shared memory multi-core platforms indicates that the proposed algorithm, ATP, has strong scalability.展开更多
In this paper,we investigate a parallel subspace correction framework for composite convex optimization.The variables are first divided into a few blocks based on certain rules.At each iteration,the algorithms solve a...In this paper,we investigate a parallel subspace correction framework for composite convex optimization.The variables are first divided into a few blocks based on certain rules.At each iteration,the algorithms solve a suitable subproblem on each block simultaneously,construct a search direction by combining their solutions on all blocks,then identify a new point along this direction using a step size satisfying the Armijo line search condition.They are called PSCLN and PSCLO,respectively,depending on whether there are overlapping regions between two imme-diately adjacent blocks of variables.Their convergence is established under mild assumptions.We compare PSCLN and PSCLO with the parallel version of the fast iterative thresholding algorithm and the fixed-point continuation method using the Barzilai-Borwein step size and the greedy coordinate block descent method for solving the l1-regularized minimization problems.Our numerical results showthatPSCLN andPSCLOcan run fast and return solutions notworse than those from the state-of-theart algorithms on most test problems.It is also observed that the overlapping domain decomposition scheme is helpful when the data of the problem has certain special structures.展开更多
Consider the problem of minimizing the sum of two convex functions,one being smooth and the other non-smooth.In this paper,we introduce a general class of approximate proximal splitting(APS)methods for solving such mi...Consider the problem of minimizing the sum of two convex functions,one being smooth and the other non-smooth.In this paper,we introduce a general class of approximate proximal splitting(APS)methods for solving such minimization problems.Methods in the APS class include many well-known algorithms such as the proximal splitting method,the block coordinate descent method(BCD),and the approximate gradient projection methods for smooth convex optimization.We establish the linear convergence of APS methods under a local error bound assumption.Since the latter is known to hold for compressive sensing and sparse group LASSO problems,our analysis implies the linear convergence of the BCD method for these problems without strong convexity assumption.展开更多
Convex clustering,turning clustering into a convex optimization problem,has drawn wide attention.It overcomes the shortcomings of traditional clustering methods such as K-means,Density-Based Spatial Clustring of Appli...Convex clustering,turning clustering into a convex optimization problem,has drawn wide attention.It overcomes the shortcomings of traditional clustering methods such as K-means,Density-Based Spatial Clustring of Applications with Noise(DBSCAN)and hierarchical clustering that can easily fall into the local optimal solution.However,convex clustering is vulnerable to the occurrence of outlier features,as it uses the Frobenius norm to measure the distance between data points and their corresponding cluster centers and evaluate clusters.To accurately identify outlier features,this paper decomposes data into a clustering structure component and a normalized component that captures outlier features.Different from existing convex clustering evaluating features with the exact measurement,the proposed model can overcome the vast difference in the magnitude of different features and the outlier features can be efficiently identified and removed.To solve the proposed model,we design an efficient algorithm and prove the global convergence of the algorithm.Experiments on both synthetic datasets and UCI datasets demonstrate that the proposed method outperforms the compared approaches in convex clustering.展开更多
基金Project supported by the National Key R&D Program of China(No.2018YFB2101100)the National Natural Science Foundation of China(Nos.61806216 and 61702533)。
文摘The era of big data in healthcare is here, and this era will significantly improve medicine and especially oncology. However, traditional machine learning algorithms need to be promoted to solve such large-scale real-world problems due to a large amount of data that needs to be analyzed and the difficulty in solving problems with nonconvex nonlinear settings. We aim to minimize the composite of a smooth nonlinear function and a block-separable nonconvex function on a large number of block variables with inequality constraints. We propose a novel parallel first-order optimization method, called asynchronous block coordinate descent with time perturbation (ATP), which adopts a time perturbation technique that escapes from saddle points and sub-optimal local points. The details of the proposed method are presented with analyses of convergence and iteration complexity properties. Experiments conducted on real-world machine learning problems validate the efficacy of our proposed method. The experimental results demonstrate that time perturbation enables ATP to escape from saddle points and sub-optimal points, providing a promising way to handle nonconvex optimization problems with inequality constraints employing asynchronous block coordinate descent. The asynchronous parallel implementation on shared memory multi-core platforms indicates that the proposed algorithm, ATP, has strong scalability.
基金Qian Dong was supported in part by the National Natural Science Foundation of China(Nos.11331012,11321061 and 11461161005)Xin Liu was supported in part by the National Natural Science Foundation of China(Nos.11101409,11331012,11471325 and 11461161005)+3 种基金China 863 Program(No.2013AA122902)the National Center for Mathematics and Interdisciplinary Sciences,Chinese Academy of SciencesZai-Wen Wen was supported in part by the National Natural Science Foundation of China(Nos.11322109 and 91330202)Ya-Xiang Yuan was supported in part by the National Natural Science Foundation of China(Nos.11331012,11321061 and 11461161005).
文摘In this paper,we investigate a parallel subspace correction framework for composite convex optimization.The variables are first divided into a few blocks based on certain rules.At each iteration,the algorithms solve a suitable subproblem on each block simultaneously,construct a search direction by combining their solutions on all blocks,then identify a new point along this direction using a step size satisfying the Armijo line search condition.They are called PSCLN and PSCLO,respectively,depending on whether there are overlapping regions between two imme-diately adjacent blocks of variables.Their convergence is established under mild assumptions.We compare PSCLN and PSCLO with the parallel version of the fast iterative thresholding algorithm and the fixed-point continuation method using the Barzilai-Borwein step size and the greedy coordinate block descent method for solving the l1-regularized minimization problems.Our numerical results showthatPSCLN andPSCLOcan run fast and return solutions notworse than those from the state-of-theart algorithms on most test problems.It is also observed that the overlapping domain decomposition scheme is helpful when the data of the problem has certain special structures.
文摘Consider the problem of minimizing the sum of two convex functions,one being smooth and the other non-smooth.In this paper,we introduce a general class of approximate proximal splitting(APS)methods for solving such minimization problems.Methods in the APS class include many well-known algorithms such as the proximal splitting method,the block coordinate descent method(BCD),and the approximate gradient projection methods for smooth convex optimization.We establish the linear convergence of APS methods under a local error bound assumption.Since the latter is known to hold for compressive sensing and sparse group LASSO problems,our analysis implies the linear convergence of the BCD method for these problems without strong convexity assumption.
基金This work was supported by the National Natural Science Foundation of China(No.11771003).
文摘Convex clustering,turning clustering into a convex optimization problem,has drawn wide attention.It overcomes the shortcomings of traditional clustering methods such as K-means,Density-Based Spatial Clustring of Applications with Noise(DBSCAN)and hierarchical clustering that can easily fall into the local optimal solution.However,convex clustering is vulnerable to the occurrence of outlier features,as it uses the Frobenius norm to measure the distance between data points and their corresponding cluster centers and evaluate clusters.To accurately identify outlier features,this paper decomposes data into a clustering structure component and a normalized component that captures outlier features.Different from existing convex clustering evaluating features with the exact measurement,the proposed model can overcome the vast difference in the magnitude of different features and the outlier features can be efficiently identified and removed.To solve the proposed model,we design an efficient algorithm and prove the global convergence of the algorithm.Experiments on both synthetic datasets and UCI datasets demonstrate that the proposed method outperforms the compared approaches in convex clustering.