We study distributed optimization problems over a directed network,where nodes aim to minimize the sum of local objective functions via directed communications with neighbors.Many algorithms are designed to solve it f...We study distributed optimization problems over a directed network,where nodes aim to minimize the sum of local objective functions via directed communications with neighbors.Many algorithms are designed to solve it for synchronized or randomly activated implementation,which may create deadlocks in practice.In sharp contrast,we propose a fully asynchronous push-pull gradient(APPG) algorithm,where each node updates without waiting for any other node by using possibly delayed information from neighbors.Then,we construct two novel augmented networks to analyze asynchrony and delays,and quantify its convergence rate from the worst-case point of view.Particularly,all nodes of APPG converge to the same optimal solution at a linear rate of O(λ^(k)) if local functions have Lipschitz-continuous gradients and their sum satisfies the Polyak-?ojasiewicz condition(convexity is not required),where λ ∈(0,1) is explicitly given and the virtual counter k increases by one when any node updates.Finally,the advantage of APPG over the synchronous counterpart and its linear speedup efficiency are numerically validated via a logistic regression problem.展开更多
This work is about a splitting method for solving a nonconvex nonseparable optimization problem with linear constraints,where the objective function consists of two separable functions and a coupled term.First,based o...This work is about a splitting method for solving a nonconvex nonseparable optimization problem with linear constraints,where the objective function consists of two separable functions and a coupled term.First,based on the ideas from Bregman distance and Peaceman–Rachford splitting method,the Bregman Peaceman–Rachford splitting method with different relaxation factors for the multiplier is proposed.Second,the global and strong convergence of the proposed algorithm are proved under general conditions including the region of the two relaxation factors as well as the crucial Kurdyka–Łojasiewicz property.Third,when the associated Kurdyka–Łojasiewicz property function has a special structure,the sublinear and linear convergence rates of the proposed algorithm are guaranteed.Furthermore,some preliminary numerical results are shown to indicate the effectiveness of the proposed algorithm.展开更多
基金Supported by National Natural Science Foundation of China(62033006,62203254)。
文摘We study distributed optimization problems over a directed network,where nodes aim to minimize the sum of local objective functions via directed communications with neighbors.Many algorithms are designed to solve it for synchronized or randomly activated implementation,which may create deadlocks in practice.In sharp contrast,we propose a fully asynchronous push-pull gradient(APPG) algorithm,where each node updates without waiting for any other node by using possibly delayed information from neighbors.Then,we construct two novel augmented networks to analyze asynchrony and delays,and quantify its convergence rate from the worst-case point of view.Particularly,all nodes of APPG converge to the same optimal solution at a linear rate of O(λ^(k)) if local functions have Lipschitz-continuous gradients and their sum satisfies the Polyak-?ojasiewicz condition(convexity is not required),where λ ∈(0,1) is explicitly given and the virtual counter k increases by one when any node updates.Finally,the advantage of APPG over the synchronous counterpart and its linear speedup efficiency are numerically validated via a logistic regression problem.
基金supported by the National Natural Science Foundation of China(No.12171106)the Natural Science Foundation of Guangxi Province(Nos.2020GXNSFDA238017 and 2018GXNSFFA281007).
文摘This work is about a splitting method for solving a nonconvex nonseparable optimization problem with linear constraints,where the objective function consists of two separable functions and a coupled term.First,based on the ideas from Bregman distance and Peaceman–Rachford splitting method,the Bregman Peaceman–Rachford splitting method with different relaxation factors for the multiplier is proposed.Second,the global and strong convergence of the proposed algorithm are proved under general conditions including the region of the two relaxation factors as well as the crucial Kurdyka–Łojasiewicz property.Third,when the associated Kurdyka–Łojasiewicz property function has a special structure,the sublinear and linear convergence rates of the proposed algorithm are guaranteed.Furthermore,some preliminary numerical results are shown to indicate the effectiveness of the proposed algorithm.
