有向网络中基于动量方法的加速行随机优化

Accelerated Row-Stochastic Optimization over Directed Networks Based on Momentum Methods

本文研究了有向网络下的分布式凸优化问题,并在现有分布式算法的基础上提出了一种新颖的分布式动量加速算法,叫做ARNH。该算法采用行随机矩阵和异构步长,有效克服网络不平衡性的同时提高了网络灵活性。此外,为了实现更快的收敛速率,ARNH采用Nesterov梯度法和Heavy-Ball法相结合的双加速机制。在局部目标函数可微且强凸的假设下,本文证明通过选取合适的步长和动量参数,算法可使节点状态渐近收敛到全局最优解。最后,在仿真实验中将ARNH与相关算法进行性能比较,验证了新算法的优越性。

In this paper, we study the distributed convex optimization problems over directed networks, and propose a novel distributed momentum acceleration algorithm called ARNH based on the existing distributed algorithms. ARNH uses row-stochastic matrix and heterogeneous step size, which effec-tively overcomes the network imbalance and improves the network flexibility. Furthermore, in or-der to achieve faster convergence, ARNH employs a double acceleration mechanism combining Nesterov gradient method and heavy-ball method. Under the assumption that the local objective function is differentiable and strongly convex, it is proved that the node state can be asymptotically converged to the global optimal solution by choosing appropriate step-sizes and momentum pa-rameters. Finally, the superiority of the new algorithm is verified by comparing the performance of ARNH with related algorithms in simulation experiments.