随机Bregman ADMM及其在训练具有离散结构的支持向量机中的应用

针对具有多块可分结构的非凸优化问题提出了一类新的随机Bregman交替方向乘子法,在周期更新规则下,证明了该算法的渐进收敛性;在随机更新的规则下,几乎确定的渐进收敛性得以证明。数值实验结果表明,该算法可有效训练具有离散结构的支持向量机。

Convergence of Bregman Alternating Direction Method of Multipliers for Nonseparable Nonconvex Objective with Linear Constraints

In this paper, our focus lies on addressing a two-block linearly constrained nonseparable nonconvex optimization problem with coupling terms. The most classical algorithm, the alternating direction method of multipliers (ADMM), is employed to solve such problems typically, which still requires the assumption of the gradient Lipschitz continuity condition on the objective function to ensure overall convergence from the current knowledge. However, many practical applications do not adhere to the conditions of smoothness. In this study, we justify the convergence of variant Bregman ADMM for the problem with coupling terms to circumvent the issue of the global Lipschitz continuity of the gradient. We demonstrate that the iterative sequence generated by our approach converges to a critical point of the issue when the corresponding function fulfills the Kurdyka-Lojasiewicz inequality and certain assumptions apply. In addition, we illustrate the convergence rate of the algorithm.

具有线性化技术的三块非凸不可分优化问题BregmanADMM收敛性分析

交替方向乘子法是求解两块可分离凸优化问题的有效方法,但是对于三块不可分的非凸优化问题的交替方向乘子法的收敛性可能无法保证.该文主要研究的是用线性化广义Bregman交替方向乘子法(L-G-BADMM)求解目标函数是三块不可分的非凸极小化问题的收敛性分析.在适当假设条件下,对算法中子问题进行求解并构建满足Kurdyka-Lojasiewicz性质的效益函数,经过理论证明可以得到该算法的收敛性.

非凸非光滑优化问题的惯性Bregman ADMM的收敛性分析

【目的】针对具有可分结构的非凸非光滑优化问题,提出一种内置惯性Bregman交替方向乘子法。【方法】为了加快算法的收敛速度,在Bregman交替方向乘子法的框架下,对子问题中的Bregman度量内置惯性项。【结果】在生成的点列有界的条件下,利用Kurdyka-Lojasiewicz性质,证明了算法的渐进收敛性。【结论】数值实验结果表明了该算法的有效性。