A Note on the Complexity of Proximal Iterative Hard Thresholding Algorithm

The iterative hard thresholding(IHT)algorithm is a powerful and efficient algorithm for solving l_(0)-regularized problems and inspired many applications in sparse-approximation and image-processing fields.Recently,some convergence results are established for the proximal scheme of IHT,namely proximal iterative hard thresholding(PIHT)algorithm(Blumensath and Davies,in J Fourier Anal Appl 14:629–654,2008;Hu et al.,Methods 67:294–303,2015;Lu,Math Program 147:125–154,2014;Trzasko et al.,IEEE/SP 14th Workshop on Statistical Signal Processing,2007)on solving the related l_(0)-optimization problems.However,the complexity analysis for the PIHT algorithm is not well explored.In this paper,we aim to provide some complexity estimations for the PIHT sequences.In particular,we show that the complexity of the sequential iterate error is at o(1/k).Under the assumption that the objective function is composed of a quadratic convex function and l_(0)regularization,we show that the PIHT algorithm has R-linear convergence rate.Finally,we illustrate some applications of this algorithm for compressive sensing reconstruction and sparse learning and validate the estimated error bounds.

Geophysical data sparse reconstruction based on L0-norm minimization

Missing data are a problem in geophysical surveys, and interpolation and reconstruction of missing data is part of the data processing and interpretation. Based on the sparseness of the geophysical data or the transform domain, we can improve the accuracy and stability of the reconstruction by transforming it to a sparse optimization problem. In this paper, we propose a mathematical model for the sparse reconstruction of data based on the LO-norm minimization. Furthermore, we discuss two types of the approximation algorithm for the LO- norm minimization according to the size and characteristics of the geophysical data： namely, the iteratively reweighted least-squares algorithm and the fast iterative hard thresholding algorithm. Theoretical and numerical analysis showed that applying the iteratively reweighted least-squares algorithm to the reconstruction of potential field data exploits its fast convergence rate, short calculation time, and high precision, whereas the fast iterative hard thresholding algorithm is more suitable for processing seismic data, moreover, its computational efficiency is better than that of the traditional iterative hard thresholding algorithm.

Image reconstruction from few views by l_0-norm optimization

In the medical computer tomography （CT） field, total variation （TV）, which is the l1-norm of the discrete gradient transform （DGT）, is widely used as regularization based on the compressive sensing （CS） theory. To overcome the TV model＇s disadvantageous tendency of uniformly penalizing the image gradient and over smoothing the low-contrast structures, an iterative algorithm based on the l0-norm optimization of the DGT is proposed. In order to rise to the challenges introduced by the l0-norm DGT, the algorithm uses a pseudo-inverse transform of DGT and adapts an iterative hard thresholding （IHT） algorithm, whose convergence and effective efficiency have been theoretically proven. The simulation demonstrates our conclusions and indicates that the algorithm proposed in this paper can obviously improve the reconstruction quality.

Seislet transform denoising based on total variation minimization

Attenuation of noise is a persistent problem in seismic exploration. The authors use conventional denoising method to remove noise which may cause vibration near the discontinuity called pseudo-Gibbs artifact.In order to remove the artifact,the study proposed a method combining the seislet transform and total variation minimization. Firstly,the data are converted into the seislet transform domain. Secondly,the hard threshold was used for eliminating the noise and keep useful signal,which is the initial input for the next step. Finally,total variation minimization dealed with denoised data to recover boundary information and further eliminated the noise. Synthetic data examples show that the method has feasibility in eliminating random noise and protecting detailed signal,and also shows better results than the classic f-x deconvolution. The field data example also shows effective in practice. It can remove the noise and preserve the discontinuity signal at the same time.

Seismic data denoising based on data-driven tight frame dictionary learning method

Because of various complicated factors in seismic data collection,the random noise of seismic data is too difficult to avoid.This random noise reduces the quality of seismic data and increases the difficulty of seismic data processing and interpretation.Improving the denoising technology is significant.In order to improve seismic data denoising result,a novel method named data-driven tight frame(DDTF)is introduced in this paper.First,we get the sparse coefficients of seismic data with noise by DDTF.Then we remove the smaller sparse coefficient by using the hard threshold function.Finally,we get the denoised seismic data by inverse transform.Furthermore,the DDTF is compared with curvelet transform in the stimulation and practical seismic data experiments to validate its performance.DDTF can raise the signal-to-noise ratio of seismic data denoising and protect the effective signal well.

Adaptive projected gradient thresholding methods for constrained l0problems

In this paper, we propose and analyze adaptive projected gradient thresholding(APGT) methods for finding sparse solutions of the underdetermined linear systems with equality and box constraints. The general convergence will be demonstrated, and in addition, the bound of the number of iterations is established in some special cases. Under suitable assumptions, it is proved that any accumulation point of the sequence generated by the APGT methods is a local minimizer of the underdetermined linear system. Moreover, the APGT methods, under certain conditions, can find all s-sparse solutions for accurate measurement cases and guarantee the stability and robustness for flawed measurement cases. Numerical examples are presented to show the accordance with theoretical results in compressed sensing and verify high out-of-sample performance in index tracking.

Improved RIP-based bounds for guaranteed performance of two compressed sensing algorithms

Iterative hard thresholding(IHT)and compressive sampling matching pursuit(CoSaMP)are two mainstream compressed sensing algorithms using the hard thresholding operator.The guaranteed performance of the two algorithms for signal recovery was mainly analyzed in terms of the restricted isometry property(RIP)of sensing matrices.At present,the best known bound using the RIP of order 3k for guaranteed performance of IHT(with the unit stepsize)isδ3k<1/√3≈0.5774,and the bound for CoSaMP using the RIP of order 4k isδ4k<0.4782.A fundamental question in this area is whether such theoretical results can be further improved.The purpose of this paper is to affirmatively answer this question and to rigorously show that the abovementioned RIP bound for guaranteed performance of IHT can be significantly improved toδ3k<(√5−1)/2≈0.618,and the bound for CoSaMP can be improved toδ4k<0.5102.

Efficient Distributed Estimation of High-dimensional Sparse Precision Matrix for Transelliptical Graphical Models

In this paper,distributed estimation of high-dimensional sparse precision matrix is proposed based on the debiased D-trace loss penalized lasso and the hard threshold method when samples are distributed into different machines for transelliptical graphical models.At a certain level of sparseness,this method not only achieves the correct selection of non-zero elements of sparse precision matrix,but the error rate can be comparable to the estimator in a non-distributed setting.The numerical results further prove that the proposed distributed method is more effective than the usual average method.