Accelerated Stochastic Peaceman–Rachford Method for Empirical Risk Minimization

This work is devoted to studying an accelerated stochastic Peaceman–Rachford splitting method(AS-PRSM)for solving a family of structural empirical risk minimization problems.The objective function to be optimized is the sum of a possibly nonsmooth convex function and a finite sum of smooth convex component functions.The smooth subproblem in AS-PRSM is solved by a stochastic gradient method using variance reduction technique and accelerated techniques,while the possibly nonsmooth subproblem is solved by introducing an indefinite proximal term to transform its solution into a proximity operator.By a proper choice for the involved parameters,we show that AS-PRSM converges in a sublinear convergence rate measured by the function value residual and constraint violation in the sense of expectation and ergodic.Preliminary experiments on testing the popular graph-guided fused lasso problem in machine learning and the 3D CT reconstruction problem in medical image processing show that the proposed AS-PRSM is very efficient.

The key theorem and the bounds on the rate of uniform convergence of learning theory on Sugeno measure space

Some properties of Sugeno measure are further discussed, which is a kind of typical nonadditive measure. The definitions and properties of gλ random variable and its distribution function, expected value, and variance are then presented. Markov inequality, Chebyshev＇s inequality and the Khinchine＇s Law of Large Numbers on Sugeno measure space are also proven. Furthermore, the concepts of empirical risk functional, expected risk functional and the strict consistency of ERM principle on Sugeno measure space are proposed. According to these properties and concepts, the key theorem of learning theory, the bounds on the rate of convergence of learning process and the relations between these bounds and capacity of the set of functions on Sugeno measure space are given.