Fast alternating direction method of multipliers for total-variation-based image restoration

A novel algorithm, i.e. the fast alternating direction method of multipliers （ADMM）, is applied to solve the classical total-variation （ TV ）-based model for image reconstruction. First, the TV-based model is reformulated as a linear equality constrained problem where the objective function is separable. Then, by introducing the augmented Lagrangian function, the two variables are alternatively minimized by the Gauss-Seidel idea. Finally, the dual variable is updated. Because the approach makes full use of the special structure of the problem and decomposes the original problem into several low-dimensional sub-problems, the per iteration computational complexity of the approach is dominated by two fast Fourier transforms. Elementary experimental results indicate that the proposed approach is more stable and efficient compared with some state-of-the-art algorithms.

Image reconstruction based on total-variation minimization and alternating direction method in linear scan computed tomography

Linear scan computed tomography （LCT） is of great benefit to online industrial scanning and security inspection due to its characteristics of straight-line source trajectory and high scanning speed. However, in practical applications of LCT, there are challenges to image reconstruction due to limited-angle and insufficient data. In this paper, a new reconstruction algorithm based on total-variation （TV） minimization is developed to reconstruct images from limited-angle and insufficient data in LCT. The main idea of our approach is to reformulate a TV problem as a linear equality constrained problem where the objective function is separable, and then minimize its augmented Lagrangian function by using alternating direction method （ADM） to solve subproblems. The proposed method is robust and efficient in the task of reconstruction by showing the convergence of ADM. The numerical simulations and real data reconstructions show that the proposed reconstruction method brings reasonable performance and outperforms some previous ones when applied to an LCT imaging problem.

Reconstruction of electrical capacitance tomography images based on fast linearized alternating direction method of multipliers for two-phase flow system

Electrical capacitance tomography(ECT)has been applied to two-phase flow measurement in recent years.Image reconstruction algorithms play an important role in the successful applications of ECT.To solve the ill-posed and nonlinear inverse problem of ECT image reconstruction,a new ECT image reconstruction method based on fast linearized alternating direction method of multipliers(FLADMM)is proposed in this paper.On the basis of theoretical analysis of compressed sensing(CS),the data acquisition of ECT is regarded as a linear measurement process of permittivity distribution signal of pipe section.A new measurement matrix is designed and L1 regularization method is used to convert ECT inverse problem to a convex relaxation problem which contains prior knowledge.A new fast alternating direction method of multipliers which contained linearized idea is employed to minimize the objective function.Simulation data and experimental results indicate that compared with other methods,the quality and speed of reconstructed images are markedly improved.Also,the dynamic experimental results indicate that the proposed algorithm can ful fill the real-time requirement of ECT systems in the application.

Nested Alternating Direction Method of Multipliers to Low-Rank and Sparse-Column Matrices Recovery

The task of dividing corrupted-data into their respective subspaces can be well illustrated,both theoretically and numerically,by recovering low-rank and sparse-column components of a given matrix.Generally,it can be characterized as a matrix and a 2,1-norm involved convex minimization problem.However,solving the resulting problem is full of challenges due to the non-smoothness of the objective function.One of the earliest solvers is an 3-block alternating direction method of multipliers(ADMM)which updates each variable in a Gauss-Seidel manner.In this paper,we present three variants of ADMM for the 3-block separable minimization problem.More preciously,whenever one variable is derived,the resulting problems can be regarded as a convex minimization with 2 blocks,and can be solved immediately using the standard ADMM.If the inner iteration loops only once,the iterative scheme reduces to the ADMM with updates in a Gauss-Seidel manner.If the solution from the inner iteration is assumed to be exact,the convergence can be deduced easily in the literature.The performance comparisons with a couple of recently designed solvers illustrate that the proposed methods are effective and competitive.

Linearized Proximal Alternating Direction Method of Multipliers for Parallel Magnetic Resonance Imaging

In this study, we propose a linearized proximal alternating direction method with variable stepsize for solving total variation image reconstruction problems. Our method uses a linearized technique and the proximal function such that the closed form solutions of the subproblem can be easily derived.In the subproblem, we apply a variable stepsize, that is like Barzilai-Borwein stepsize, to accelerate the algorithm. Numerical results with parallel magnetic resonance imaging demonstrate the efficiency of the proposed algorithm.

Distributed MPC for Reconfigurable Architecture Systems via Alternating Direction Method of Multipliers

This paper investigates the distributed model predictive control(MPC)problem of linear systems where the network topology is changeable by the way of inserting new subsystems,disconnecting existing subsystems,or merely modifying the couplings between different subsystems.To equip live systems with a quick response ability when modifying network topology,while keeping a satisfactory dynamic performance,a novel reconfiguration control scheme based on the alternating direction method of multipliers(ADMM)is presented.In this scheme,the local controllers directly influenced by the structure realignment are redesigned in the reconfiguration control.Meanwhile,by employing the powerful ADMM algorithm,the iterative formulas for solving the reconfigured optimization problem are obtained,which significantly accelerate the computation speed and ensure a timely output of the reconfigured optimal control response.Ultimately,the presented reconfiguration scheme is applied to the level control of a benchmark four-tank plant to illustrate its effectiveness and main characteristics.

Multichannel Blind CT Image Restoration via Variable Splitting and Alternating Direction Method

Computed tomography(CT) blurring caused by point spread function leads to errors in quantification and visualization. In this paper, multichannel blind CT image restoration is proposed to overcome the effect of point spread function. The main advantage from multichannel blind CT image restoration is to exploit the diversity and redundancy of information in different acquisitions. The proposed approach is based on a variable splitting to obtain an equivalent constrained optimization formulation, which is addressed with the alternating direction method of multipliers and simply implemented in the Fourier domain. Numerical experiments illustrate that our method obtains a higher average gain value of at least 1.21 d B in terms of Q metric than the other methods, and it requires only 7 iterations of alternating minimization to obtain a fast convergence.

Distributed Alternating Direction Method of Multipliers for Multi-Objective Optimization

In this paper, a distributed algorithm is proposed to solve a kind of multi-objective optimization problem based on the alternating direction method of multipliers. Compared with the centralized algorithms, this algorithm does not need a central node. Therefore, it has the characteristics of low communication burden and high privacy. In addition, numerical experiments are provided to validate the effectiveness of the proposed algorithm.

Stochastic Accelerated Alternating Direction Method of Multipliers for Hedging Communication Noise in Combined Heat and Power Dispatch

Combined heat and power dispatch(CHPD)opens a new window for increasing operational flexibility and reducing wind power curtailment.Electric power and district heating systems are independently controlled by different system operators;therefore,a decentralized solution paradigm is necessary for CHPD,in which only minor boundary information is required to be exchanged via a communication network.However,a nonideal communication environment with noise could lead to divergence or incorrect solutions of decentralized algorithms.To bridge this gap,this paper proposes a stochastic accelerated alternating direction method of multipliers(SA-ADMM)for hedging communication noise in CHPD.This algorithm provides a general framework to address more types of constraint sets and separable objective functions than the existing stochastic ADMM.Different from the single noise sources considered in the existing stochastic approximation methods,communication noise from multiple sources is addressed in both the local calculation and the variable update stages.Case studies of two test systems validate the effectiveness and robustness of the proposed SAADMM.

A Bregman-style Partially Symmetric Alternating Direction Method of Multipliers for Nonconvex Multi-block Optimization

The alternating direction method of multipliers(ADMM)is one of the most successful and powerful methods for separable minimization optimization.Based on the idea of symmetric ADMM in two-block optimization,we add an updating formula for the Lagrange multiplier without restricting its position for multiblock one.Then,combining with the Bregman distance,in this work,a Bregman-style partially symmetric ADMM is presented for nonconvex multi-block optimization with linear constraints,and the Lagrange multiplier is updated twice with different relaxation factors in the iteration scheme.Under the suitable conditions,the global convergence,strong convergence and convergence rate of the presented method are analyzed and obtained.Finally,some preliminary numerical results are reported to support the correctness of the theoretical assertions,and these show that the presented method is numerically effective.

A New Stopping Criterion for Eckstein and Bertsekas’s Generalized Alternating Direction Method of Multipliers

In this paper,we propose a new stopping criterion for Eckstein and Bertsekas’s generalized alternating direction method of multipliers.The stopping criterion is easy to verify,and the computational cost is much less than the classical stopping criterion in the highly influential paper by Boyd et al.(Found Trends Mach Learn 3(1):1–122,2011).

A Symmetric Linearized Alternating Direction Method of Multipliers for a Class of Stochastic Optimization Problems

Alternating direction method of multipliers(ADMM)receives much attention in the recent years due to various demands from machine learning and big data related optimization.In 2013,Ouyang et al.extend the ADMM to the stochastic setting for solving some stochastic optimization problems,inspired by the structural risk minimization principle.In this paper,we consider a stochastic variant of symmetric ADMM,named symmetric stochastic linearized ADMM(SSL-ADMM).In particular,using the framework of variational inequality,we analyze the convergence properties of SSL-ADMM.Moreover,we show that,with high probability,SSL-ADMM has O((ln N)·N^(-1/2))constraint violation bound and objective error bound for convex problems,and has O((ln N)^(2)·N^(-1))constraint violation bound and objective error bound for strongly convex problems,where N is the iteration number.Symmetric ADMM can improve the algorithmic performance compared to classical ADMM,numerical experiments for statistical machine learning show that such an improvement is also present in the stochastic setting.

Impact Force Localization and Reconstruction via ADMM-based Sparse Regularization Method

In practice,simultaneous impact localization and time history reconstruction can hardly be achieved,due to the illposed and under-determined problems induced by the constrained and harsh measuring conditions.Although l_(1) regularization can be used to obtain sparse solutions,it tends to underestimate solution amplitudes as a biased estimator.To address this issue,a novel impact force identification method with l_(p) regularization is proposed in this paper,using the alternating direction method of multipliers(ADMM).By decomposing the complex primal problem into sub-problems solvable in parallel via proximal operators,ADMM can address the challenge effectively.To mitigate the sensitivity to regularization parameters,an adaptive regularization parameter is derived based on the K-sparsity strategy.Then,an ADMM-based sparse regularization method is developed,which is capable of handling l_(p) regularization with arbitrary p values using adaptively-updated parameters.The effectiveness and performance of the proposed method are validated on an aircraft skin-like composite structure.Additionally,an investigation into the optimal p value for achieving high-accuracy solutions via l_(p) regularization is conducted.It turns out that l_(0.6)regularization consistently yields sparser and more accurate solutions for impact force identification compared to the classic l_(1) regularization method.The impact force identification method proposed in this paper can simultaneously reconstruct impact time history with high accuracy and accurately localize the impact using an under-determined sensor configuration.

Optimization of Random Feature Method in the High-Precision Regime

Machine learning has been widely used for solving partial differential equations(PDEs)in recent years,among which the random feature method(RFM)exhibits spectral accuracy and can compete with traditional solvers in terms of both accuracy and efficiency.Potentially,the optimization problem in the RFM is more difficult to solve than those that arise in traditional methods.Unlike the broader machine-learning research,which frequently targets tasks within the low-precision regime,our study focuses on the high-precision regime crucial for solving PDEs.In this work,we study this problem from the following aspects:(i)we analyze the coeffcient matrix that arises in the RFM by studying the distribution of singular values;(ii)we investigate whether the continuous training causes the overfitting issue;(ii)we test direct and iterative methods as well as randomized methods for solving the optimization problem.Based on these results,we find that direct methods are superior to other methods if memory is not an issue,while iterative methods typically have low accuracy and can be improved by preconditioning to some extent.

Superconvergence of Direct Discontinuous Galerkin Methods:Eigen-structure Analysis Based on Fourier Approach

This paper investigates superconvergence properties of the direct discontinuous Galerkin(DDG)method with interface corrections and the symmetric DDG method for diffusion equations.We apply the Fourier analysis technique to symbolically compute eigenvalues and eigenvectors of the amplification matrices for both DDG methods with different coefficient settings in the numerical fluxes.Based on the eigen-structure analysis,we carry out error estimates of the DDG solutions,which can be decomposed into three parts:(i)dissipation errors of the physically relevant eigenvalue,which grow linearly with the time and are of order 2k for P^(k)(k=2,3)approximations;(ii)projection error from a special projection of the exact solution,which is decreasing over the time and is related to the eigenvector corresponding to the physically relevant eigenvalue;(iii)dissipative errors of non-physically relevant eigenvalues,which decay exponentially with respect to the spatial mesh sizeΔx.We observe that the errors are sensitive to the choice of the numerical flux coefficient for even degree P^(2)approximations,but are not for odd degree P^(3)approximations.Numerical experiments are provided to verify the theoretical results.

A proximal point algorithm revisit on the alternating direction method of multipliers

The alternating direction method of multipliers(ADMM)is a benchmark for solving convex programming problems with separable objective functions and linear constraints.In the literature it has been illustrated as an application of the proximal point algorithm(PPA)to the dual problem of the model under consideration.This paper shows that ADMM can also be regarded as an application of PPA to the primal model with a customized choice of the proximal parameter.This primal illustration of ADMM is thus complemental to its dual illustration in the literature.This PPA revisit on ADMM from the primal perspective also enables us to recover the generalized ADMM proposed by Eckstein and Bertsekas easily.A worst-case O(1/t)convergence rate in ergodic sense is established for a slight extension of Eckstein and Bertsekas’s generalized ADMM.

Decentralized Demand Management Based on Alternating Direction Method of Multipliers Algorithm for Industrial Park with CHP Units and Thermal Storage

This paper proposes a decentralized demand management approach to reduce the energy bill of industrial park and improve its economic gains.A demand management model for industrial park considering the integrated demand response of combined heat and power(CHP)units and thermal storage is firstly proposed.Specifically,by increasing the electricity outputs of CHP units during peak-load periods,not only the peak demand charge but also the energy charge can be reduced.The thermal storage can efficiently utilize the waste heat provided by CHP units and further increase the flexibility of CHP units.The heat dissipation of thermal storage,thermal delay effect,and heat losses of heat pipelines are considered for ensuring reliable solutions to the industrial park.The proposed model is formulated as a multi-period alternating current(AC)optimal power flow problem via the second-order conic programming formulation.The alternating direction method of multipliers(ADMM)algorithm is used to compute the proposed demand management model in a distributed manner,which can protect private data of all participants while achieving solutions with high quality.Numerical case studies validate the effectiveness of the proposed demand management approach in reducing peak demand charge,and the performance of the ADMM-based decentralized computation algorithm in deriving the same optimal results of demand management as the centralized approach is also validated.

A Survey on Some Recent Developments of Alternating Direction Method of Multipliers

Recently, alternating direction method of multipliers (ADMM) attracts much attentions from various fields and there are many variant versions tailored for differentmodels. Moreover, its theoretical studies such as rate of convergence and extensionsto nonconvex problems also achieve much progress. In this paper, we give a surveyon some recent developments of ADMM and its variants.

Alternating Direction Method of Multipliers for Sparse Principal Component Analysis

We consider a convex relaxation of sparse principal component analysisproposed by d' Aspremont et al. (SIAM Rev. 49:434 448, 2007). This convex relax-ation is a nonsmooth semidefinite programming problem in which the ξ1 norm of thedesired matrix is imposed in either the objective function or the constraint to improvethe sparsity of the resulting matrix. The sparse principal component is obtained by arank- one decomposition of the resulting sparse matrix. We propose an alternating di-rection method based on a variable-splitting technique and an augmented I agrangianframework for solving this nonsmooth semidefinite programming problem. In con-trast to the first-order method proposed in d' Aspremont et al. (SIAM Rev. 49:434448, 2007), which solves approximately the dual problem of the original semidefiniteprogramming problem, our method deals with the primal problem directly and solvesit exactly, which guarantees that the resulting matrix is a sparse matrix. A globalconvergence result is established for the proposed method. Numerical results on bothsynthetic problems and the real applications from classification of text data and senatevoting data are reported to demonstrate the efficacy of our method.

Adaptive Linearized Alternating Direction Method of Multipliers for Non-Convex Compositely Regularized Optimization Problems

We consider a wide range of non-convex regularized minimization problems, where the non-convex regularization term is composite with a linear function engaged in sparse learning. Recent theoretical investigations have demonstrated their superiority over their convex counterparts. The computational challenge lies in the fact that the proximal mapping associated with non-convex regularization is not easily obtained due to the imposed linear composition. Fortunately, the problem structure allows one to introduce an auxiliary variable and reformulate it as an optimization problem with linear constraints, which can be solved using the Linearized Alternating Direction Method of Multipliers （LADMM）. Despite the success of LADMM in practice, it remains unknown whether LADMM is convergent in solving such non-convex compositely regularized optimizations. In this research, we first present a detailed convergence analysis of the LADMM algorithm for solving a non-convex compositely regularized optimization problem with a large class of non-convex penalties. Furthermore, we propose an Adaptive LADMM （AdaLADMM） algorithm with a line-search criterion. Experimental results on different genres of datasets validate the efficacy of the proposed algorithm.