Fast alternating direction method of multipliers for total-variation-based image restoration

A novel algorithm, i.e. the fast alternating direction method of multipliers （ADMM）, is applied to solve the classical total-variation （ TV ）-based model for image reconstruction. First, the TV-based model is reformulated as a linear equality constrained problem where the objective function is separable. Then, by introducing the augmented Lagrangian function, the two variables are alternatively minimized by the Gauss-Seidel idea. Finally, the dual variable is updated. Because the approach makes full use of the special structure of the problem and decomposes the original problem into several low-dimensional sub-problems, the per iteration computational complexity of the approach is dominated by two fast Fourier transforms. Elementary experimental results indicate that the proposed approach is more stable and efficient compared with some state-of-the-art algorithms.

Reconstruction of electrical capacitance tomography images based on fast linearized alternating direction method of multipliers for two-phase flow system

Electrical capacitance tomography(ECT)has been applied to two-phase flow measurement in recent years.Image reconstruction algorithms play an important role in the successful applications of ECT.To solve the ill-posed and nonlinear inverse problem of ECT image reconstruction,a new ECT image reconstruction method based on fast linearized alternating direction method of multipliers(FLADMM)is proposed in this paper.On the basis of theoretical analysis of compressed sensing(CS),the data acquisition of ECT is regarded as a linear measurement process of permittivity distribution signal of pipe section.A new measurement matrix is designed and L1 regularization method is used to convert ECT inverse problem to a convex relaxation problem which contains prior knowledge.A new fast alternating direction method of multipliers which contained linearized idea is employed to minimize the objective function.Simulation data and experimental results indicate that compared with other methods,the quality and speed of reconstructed images are markedly improved.Also,the dynamic experimental results indicate that the proposed algorithm can ful fill the real-time requirement of ECT systems in the application.

Nested Alternating Direction Method of Multipliers to Low-Rank and Sparse-Column Matrices Recovery

The task of dividing corrupted-data into their respective subspaces can be well illustrated,both theoretically and numerically,by recovering low-rank and sparse-column components of a given matrix.Generally,it can be characterized as a matrix and a 2,1-norm involved convex minimization problem.However,solving the resulting problem is full of challenges due to the non-smoothness of the objective function.One of the earliest solvers is an 3-block alternating direction method of multipliers(ADMM)which updates each variable in a Gauss-Seidel manner.In this paper,we present three variants of ADMM for the 3-block separable minimization problem.More preciously,whenever one variable is derived,the resulting problems can be regarded as a convex minimization with 2 blocks,and can be solved immediately using the standard ADMM.If the inner iteration loops only once,the iterative scheme reduces to the ADMM with updates in a Gauss-Seidel manner.If the solution from the inner iteration is assumed to be exact,the convergence can be deduced easily in the literature.The performance comparisons with a couple of recently designed solvers illustrate that the proposed methods are effective and competitive.

Distributed Alternating Direction Method of Multipliers for Multi-Objective Optimization

In this paper, a distributed algorithm is proposed to solve a kind of multi-objective optimization problem based on the alternating direction method of multipliers. Compared with the centralized algorithms, this algorithm does not need a central node. Therefore, it has the characteristics of low communication burden and high privacy. In addition, numerical experiments are provided to validate the effectiveness of the proposed algorithm.

Distributed MPC for Reconfigurable Architecture Systems via Alternating Direction Method of Multipliers

This paper investigates the distributed model predictive control(MPC)problem of linear systems where the network topology is changeable by the way of inserting new subsystems,disconnecting existing subsystems,or merely modifying the couplings between different subsystems.To equip live systems with a quick response ability when modifying network topology,while keeping a satisfactory dynamic performance,a novel reconfiguration control scheme based on the alternating direction method of multipliers(ADMM)is presented.In this scheme,the local controllers directly influenced by the structure realignment are redesigned in the reconfiguration control.Meanwhile,by employing the powerful ADMM algorithm,the iterative formulas for solving the reconfigured optimization problem are obtained,which significantly accelerate the computation speed and ensure a timely output of the reconfigured optimal control response.Ultimately,the presented reconfiguration scheme is applied to the level control of a benchmark four-tank plant to illustrate its effectiveness and main characteristics.

Impact Force Localization and Reconstruction via ADMM-based Sparse Regularization Method

In practice,simultaneous impact localization and time history reconstruction can hardly be achieved,due to the illposed and under-determined problems induced by the constrained and harsh measuring conditions.Although l_(1) regularization can be used to obtain sparse solutions,it tends to underestimate solution amplitudes as a biased estimator.To address this issue,a novel impact force identification method with l_(p) regularization is proposed in this paper,using the alternating direction method of multipliers(ADMM).By decomposing the complex primal problem into sub-problems solvable in parallel via proximal operators,ADMM can address the challenge effectively.To mitigate the sensitivity to regularization parameters,an adaptive regularization parameter is derived based on the K-sparsity strategy.Then,an ADMM-based sparse regularization method is developed,which is capable of handling l_(p) regularization with arbitrary p values using adaptively-updated parameters.The effectiveness and performance of the proposed method are validated on an aircraft skin-like composite structure.Additionally,an investigation into the optimal p value for achieving high-accuracy solutions via l_(p) regularization is conducted.It turns out that l_(0.6)regularization consistently yields sparser and more accurate solutions for impact force identification compared to the classic l_(1) regularization method.The impact force identification method proposed in this paper can simultaneously reconstruct impact time history with high accuracy and accurately localize the impact using an under-determined sensor configuration.

An Efficient Smoothing and Thresholding Image Segmentation Framework with Weighted Anisotropic-Isotropic Total Variation

In this paper,we design an efficient,multi-stage image segmentation framework that incorporates a weighted difference of anisotropic and isotropic total variation(AITV).The segmentation framework generally consists of two stages:smoothing and thresholding,thus referred to as smoothing-and-thresholding(SaT).In the first stage,a smoothed image is obtained by an AITV-regularized Mumford-Shah(MS)model,which can be solved efficiently by the alternating direction method of multipliers(ADMMs)with a closed-form solution of a proximal operator of the l_(1)-αl_(2) regularizer.The convergence of the ADMM algorithm is analyzed.In the second stage,we threshold the smoothed image by K-means clustering to obtain the final segmentation result.Numerical experiments demonstrate that the proposed segmentation framework is versatile for both grayscale and color images,effcient in producing high-quality segmentation results within a few seconds,and robust to input images that are corrupted with noise,blur,or both.We compare the AITV method with its original convex TV and nonconvex TVP(O<p<1)counterparts,showcasing the qualitative and quantitative advantages of our proposed method.

Fully Distributed Learning for Deep Random Vector Functional-Link Networks

In the contemporary era, the proliferation of information technology has led to an unprecedented surge in data generation, with this data being dispersed across a multitude of mobile devices. Facing these situations and the training of deep learning model that needs great computing power support, the distributed algorithm that can carry out multi-party joint modeling has attracted everyone’s attention. The distributed training mode relieves the huge pressure of centralized model on computer computing power and communication. However, most distributed algorithms currently work in a master-slave mode, often including a central server for coordination, which to some extent will cause communication pressure, data leakage, privacy violations and other issues. To solve these problems, a decentralized fully distributed algorithm based on deep random weight neural network is proposed. The algorithm decomposes the original objective function into several sub-problems under consistency constraints, combines the decentralized average consensus (DAC) and alternating direction method of multipliers (ADMM), and achieves the goal of joint modeling and training through local calculation and communication of each node. Finally, we compare the proposed decentralized algorithm with several centralized deep neural networks with random weights, and experimental results demonstrate the effectiveness of the proposed algorithm.

Generalized Nonconvex Low-Rank Algorithm for Magnetic Resonance Imaging (MRI) Reconstruction

In recent years,utilizing the low-rank prior information to construct a signal from a small amount of measures has attracted much attention.In this paper,a generalized nonconvex low-rank(GNLR) algorithm for magnetic resonance imaging(MRI)reconstruction is proposed,which reconstructs the image from highly under-sampled k-space data.In the algorithm,the nonconvex surrogate function replacing the conventional nuclear norm is utilized to enhance the low-rank property inherent in the reconstructed image.An alternative direction multiplier method(ADMM) is applied to solving the resulting non-convex model.Extensive experimental results have demonstrated that the proposed method can consistently recover MRIs efficiently,and outperforms the current state-of-the-art approaches in terms of higher peak signal-to-noise ratio(PSNR) and lower high-frequency error norm(HFEN) values.

MEC和区块链赋能无人机辅助的物联网资源优化

针对物联网设备部署在较偏远地区而导致的传输链路易受损或传输覆盖范围有限等问题,在此场景中引入无人机和移动边缘计算(mobile edge computing, MEC)技术,有效改善物联网设备能源供给,优化计算资源,同时提升通信覆盖范围,减少不必要的网络开销.另外,区块链技术的引入保证了数据计算卸载与交互过程中的安全性和可靠性,实现了数据共享.因此,面向无人机辅助的物联网系统提出一种融合MEC和区块链的资源分配决策方法,以实现MEC系统和区块链系统性能的最佳权衡为目标,综合考虑频谱资源和计算资源的分配,构建问题模型,并采用基于交替方向乘子(alternating direction method of multipliers, ADMM)法的分布式优化算法求解该优化问题.仿真结果表明,所提优化框架可以有效减少MEC系统的总能耗和区块链系统的计算时延.同时,所提方法具有良好的收敛性能,系统稳定性得到充分保证.

A proximal point algorithm revisit on the alternating direction method of multipliers

The alternating direction method of multipliers(ADMM)is a benchmark for solving convex programming problems with separable objective functions and linear constraints.In the literature it has been illustrated as an application of the proximal point algorithm(PPA)to the dual problem of the model under consideration.This paper shows that ADMM can also be regarded as an application of PPA to the primal model with a customized choice of the proximal parameter.This primal illustration of ADMM is thus complemental to its dual illustration in the literature.This PPA revisit on ADMM from the primal perspective also enables us to recover the generalized ADMM proposed by Eckstein and Bertsekas easily.A worst-case O(1/t)convergence rate in ergodic sense is established for a slight extension of Eckstein and Bertsekas’s generalized ADMM.

A Survey on Some Recent Developments of Alternating Direction Method of Multipliers

Recently, alternating direction method of multipliers (ADMM) attracts much attentions from various fields and there are many variant versions tailored for differentmodels. Moreover, its theoretical studies such as rate of convergence and extensionsto nonconvex problems also achieve much progress. In this paper, we give a surveyon some recent developments of ADMM and its variants.

Decentralized Demand Management Based on Alternating Direction Method of Multipliers Algorithm for Industrial Park with CHP Units and Thermal Storage

This paper proposes a decentralized demand management approach to reduce the energy bill of industrial park and improve its economic gains.A demand management model for industrial park considering the integrated demand response of combined heat and power(CHP)units and thermal storage is firstly proposed.Specifically,by increasing the electricity outputs of CHP units during peak-load periods,not only the peak demand charge but also the energy charge can be reduced.The thermal storage can efficiently utilize the waste heat provided by CHP units and further increase the flexibility of CHP units.The heat dissipation of thermal storage,thermal delay effect,and heat losses of heat pipelines are considered for ensuring reliable solutions to the industrial park.The proposed model is formulated as a multi-period alternating current(AC)optimal power flow problem via the second-order conic programming formulation.The alternating direction method of multipliers(ADMM)algorithm is used to compute the proposed demand management model in a distributed manner,which can protect private data of all participants while achieving solutions with high quality.Numerical case studies validate the effectiveness of the proposed demand management approach in reducing peak demand charge,and the performance of the ADMM-based decentralized computation algorithm in deriving the same optimal results of demand management as the centralized approach is also validated.

Adaptive Linearized Alternating Direction Method of Multipliers for Non-Convex Compositely Regularized Optimization Problems

We consider a wide range of non-convex regularized minimization problems, where the non-convex regularization term is composite with a linear function engaged in sparse learning. Recent theoretical investigations have demonstrated their superiority over their convex counterparts. The computational challenge lies in the fact that the proximal mapping associated with non-convex regularization is not easily obtained due to the imposed linear composition. Fortunately, the problem structure allows one to introduce an auxiliary variable and reformulate it as an optimization problem with linear constraints, which can be solved using the Linearized Alternating Direction Method of Multipliers （LADMM）. Despite the success of LADMM in practice, it remains unknown whether LADMM is convergent in solving such non-convex compositely regularized optimizations. In this research, we first present a detailed convergence analysis of the LADMM algorithm for solving a non-convex compositely regularized optimization problem with a large class of non-convex penalties. Furthermore, we propose an Adaptive LADMM （AdaLADMM） algorithm with a line-search criterion. Experimental results on different genres of datasets validate the efficacy of the proposed algorithm.

An LQP-Based Symmetric Alternating Direction Method of Multipliers with Larger Step Sizes

Symmetric alternating directionmethod of multipliers(ADMM)is an efficient method for solving a class of separable convex optimization problems.This method updates the Lagrange multiplier twice with appropriate step sizes at each iteration.However,such step sizes were conservatively shrunk to guarantee the convergence in recent studies.In this paper,we are devoted to seeking larger step sizes whenever possible.The logarithmic-quadratic proximal(LQP)terms are applied to regularize the symmetric ADMM subproblems,allowing the constrained subproblems to then be converted to easier unconstrained ones.Theoretically,we prove the global convergence of such LQP-based symmetric ADMM by specifying a larger step size domain.Moreover,the numerical results on a traffic equilibrium problem are reported to demonstrate the advantage of the method with larger step sizes.

Relaxed Alternating Direction Method of Multipliers for Hedging Communication Packet Loss in Integrated Electrical and Heating System

Integrated electrical and heating systems(IEHSs)are promising for increasing the flexibility of power systems by exploiting the heat energy storage of pipelines.With the recent development of advanced communication technology,distributed optimization is employed in the coordination of IEHSs to meet the practical requirement of information privacy between different system operators.Existing studies on distributed optimization algorithms for IEHSs have seldom addressed packet loss during the process of information exchange.In this paper,a distributed paradigm is proposed for coordinating the operation of an IEHS considering communication packet loss.The relaxed alternating direction method of multipliers(R-ADMM)is derived by applying Peaceman-Rachford splitting to the Lagrangian dual of the primal problem.The proposed method is tested using several test systems in a lossy communication and transmission environment.Simulation results indicate the effectiveness and robustness of the proposed R-ADMM algorithm.

An Alternating Direction Method of Multipliers for MCP-penalized Regression with High-dimensional Data

The minimax concave penalty （MCP） has been demonstrated theoretically and practical- ly to be effective in nonconvex penalization for variable selection and parameter estimation. In this paper, we develop an efficient alternating direction method of multipliers （ADMM） with continuation algorithm for solving the MCP-penalized least squares problem in high dimensions. Under some mild conditions, we study the convergence properties and the Karush-Kuhn-Tucker （KKT） optimality con- ditions of the proposed method. A high-dimensional BIC is developed to select the optimal tuning parameters. Simulations and a real data example are presented to illustrate the efficiency and accuracy of the proposed method.

Two-stage ADMM-based distributed optimal reactive power control method for wind farms considering wake effects

Since the connection of small-scale wind farms to distribution networks,power grid voltage stability has been reduced with increasing wind penetration in recent years,owing to the variable reactive power consumption of wind generators.In this study,a two-stage reactive power optimization method based on the alternating direction method of multipliers(ADMM)algorithm is proposed for achieving optimal reactive power dispatch in wind farm-integrated distribution systems.Unlike existing optimal reactive power control methods,the proposed method enables distributed reactive power flow optimization with a two-stage optimization structure.Furthermore,under the partition concept,the consensus protocol is not needed to solve the optimization problems.In this method,the influence of the wake effect of each wind turbine is also considered in the control design.Simulation results for a mid-voltage distribution system based on MATLAB verified the effectiveness of the proposed method.

Active User and Data Detection for Uplink Grant-free NOMA Systems

This paper proposes some low complexity algorithms for active user detection(AUD),channel estimation(CE)and multi-user detection(MUD)in uplink non-orthogonal multiple access(NOMA)systems,including single-carrier and multi-carrier cases.In particular,we first propose a novel algorithm to estimate the active users and the channels for single-carrier based on complex alternating direction method of multipliers(ADMM),where fast decaying feature of non-zero components in sparse signal is considered.More importantly,the reliable estimated information is used for AUD,and the unreliable information will be further handled based on estimated symbol energy and total accurate or approximate number of active users.Then,the proposed algorithm for AUD in single-carrier model can be extended to multi-carrier case by exploiting the block sparse structure.Besides,we propose a low complexity MUD detection algorithm based on alternating minimization to estimate the active users’data,which avoids the Hessian matrix inverse.The convergence and the complexity of proposed algorithms are analyzed and discussed finally.Simulation results show that the proposed algorithms have better performance in terms of AUD,CE and MUD.Moreover,we can detect active users perfectly for multi-carrier NOMA system.

An Efficient Algorithm for Low Rank Matrix Restoration Problem with Unknown Noise Level

Recovering an unknown high dimensional low rank matrix from a small set of entries is widely spread in the fields of machine learning,system identification and image restoration,etc.In many practical applications,the few observations are always corrupted by noise and the noise level is also unknown.A novel model with nuclear norm and square root type estimator has been proposed,which does not rely on the knowledge or on an estimation of the standard deviation of the noise.In this paper,we firstly reformulate the problem to an equivalent variable separated form by introducing an auxiliary variable.Then we propose an efficient alternating direction method of multipliers(ADMM)for solving it.Both of resulting subproblems admit an explicit solution,which makes our algorithm have a cheap computing.Finally,the numerical results show the benefits of the model and the efficiency of the proposed method.