SAR image de-noising based on texture strength and weighted nuclear norm minimization

As synthetic aperture radar（SAR） has been widely used nearly in every field, SAR image de-noising became a very important research field. A new SAR image de-noising method based on texture strength and weighted nuclear norm minimization（WNNM） is proposed. To implement blind de-noising, the accurate estimation of noise variance is very important. So far, it is still a challenge to estimate SAR image noise level accurately because of the rich texture. Principal component analysis（PCA） and the low rank patches selected by image texture strength are used to estimate the noise level. With the help of noise level, WNNM can be expected to SAR image de-noising. Experimental results show that the proposed method outperforms many excellent de-noising algorithms such as Bayes least squares-Gaussian scale mixtures（BLS-GSM） method, non-local means（NLM） filtering in terms of both quantitative measure and visual perception quality.

Double Transformed Tubal Nuclear Norm Minimization for Tensor Completion

Non-convex methods play a critical role in low-rank tensor completion for their approximation to tensor rank is tighter than that of convex methods.But they usually cost much more time for calculating singular values of large tensors.In this paper,we propose a double transformed tubal nuclear norm(DTTNN)to replace the rank norm penalty in low rank tensor completion(LRTC)tasks.DTTNN turns the original non-convex penalty of a large tensor into two convex penalties of much smaller tensors,and it is shown to be an equivalent transformation.Therefore,DTTNN could take advantage of non-convex envelopes while saving time.Experimental results on color image and video inpainting tasks verify the effectiveness of DTTNN compared with state-of-the-art methods.

Identification of Continuous-Time Hammerstein System with Nuclear Norm Convex Relaxation

The nuclear norm convex relaxation method is proposed to force the rank constraint in the identification of the continuous-time( CT) Hammerstein system. The CT Hammerstein system is composed of a linear time invariant( LTI) system and a static nonlinear function( the linear part is followed by the nonlinear part). The nonlinear function is approximated by the pseudospectral basis functions, which have a better performance than Hinge functions and Radial Basis functions. After the approximation on the nonlinear function, the CT Hammerstein system has been transformed into a multiple-input single-output( MISO) linear model system with the differential pre-filters. However, the coefficients of static nonlinearity and the numerators of the linear transfer function are coupled together to challenge the parameters identification of the Hammerstein system. This problem is solved by replacing the one-rank constraint of the regularization optimization with the nuclear norm convex relaxation. Finally, a numerical example is given to verify the accuracy and the efficiency of the method.

Weighted Nuclear Norm Minimization-Based Regularization Method for Image Restoration

Regularization methods have been substantially applied in image restoration due to the ill-posedness of the image restoration problem.Different assumptions or priors on images are applied in the construction of image regularization methods.In recent years,matrix low-rank approximation has been successfully introduced in the image denoising problem and significant denoising effects have been achieved.Low-rank matrix minimization is an NP-hard problem and it is often replaced with the matrix’s weighted nuclear norm minimization(WNNM).The assumption that an image contains an extensive amount of self-similarity is the basis for the construction of the matrix low-rank approximation-based image denoising method.In this paper,we develop a model for image restoration using the sum of block matching matrices’weighted nuclear norm to be the regularization term in the cost function.An alternating iterative algorithm is designed to solve the proposed model and the convergence analyses of the algorithm are also presented.Numerical experiments show that the proposed method can recover the images much better than the existing regularization methods in terms of both recovered quantities and visual qualities.

Parallel Active Subspace Decomposition for Tensor Robust Principal Component Analysis

Tensor robust principal component analysis has received a substantial amount of attention in various fields.Most existing methods,normally relying on tensor nuclear norm minimization,need to pay an expensive computational cost due to multiple singular value decompositions at each iteration.To overcome the drawback,we propose a scalable and efficient method,named parallel active subspace decomposition,which divides the unfolding along each mode of the tensor into a columnwise orthonormal matrix(active subspace)and another small-size matrix in parallel.Such a transformation leads to a nonconvex optimization problem in which the scale of nuclear norm minimization is generally much smaller than that in the original problem.We solve the optimization problem by an alternating direction method of multipliers and show that the iterates can be convergent within the given stopping criterion and the convergent solution is close to the global optimum solution within the prescribed bound.Experimental results are given to demonstrate that the performance of the proposed model is better than the state-of-the-art methods.

LOW-RANK MATRIX COMPLETION WITH POISSON OBSERVATIONS VIA NUCLEAR NORM AND TOTAL VARIATION CONSTRAINTS

In this paper,we study the low-rank matrix completion problem with Poisson observations,where only partial entries are available and the observations are in the presence of Poisson noise.We propose a novel model composed of the Kullback-Leibler(KL)divergence by using the maximum likelihood estimation of Poisson noise,and total variation(TV)and nuclear norm constraints.Here the nuclear norm and TV constraints are utilized to explore the approximate low-rankness and piecewise smoothness of the underlying matrix,respectively.The advantage of these two constraints in the proposed model is that the low-rankness and piecewise smoothness of the underlying matrix can be exploited simultaneously,and they can be regularized for many real-world image data.An upper error bound of the estimator of the proposed model is established with high probability,which is not larger than that of only TV or nuclear norm constraint.To the best of our knowledge,this is the first work to utilize both low-rank and TV constraints with theoretical error bounds for matrix completion under Poisson observations.Extensive numerical examples on both synthetic data and real-world images are reported to corroborate the superiority of the proposed approach.

A COMPLETE CHARACTERIZATION OF THE ROBUST ISOLATED CALMNESS OF NUCLEAR NORM REGULARIZED CONVEX OPTIMIZATION PROBLEMS

In this paper, we provide a complete characterization of the robust isolated calmness of the Karush-Kuhn-Tucker （KKT） solution mapping for convex constrained optimization problems regularized by the nuclear norm function. This study is motivated by the recent work in [8], where the authors show that under the Robinson constraint qualification at a local optimal solution, the KKT solution mapping for a wide class of conic programming problems is robustly isolated calm if and only if both the second order sufficient condition （SOSC） and the strict Robinson constraint qualification （SRCQ） are satisfied. Based on the variational properties of the nuclear norm function and its conjugate, we establish the equivalence between the primal/dual SOSC and the dual/primal SRCQ. The derived results lead to several equivalent characterizations of the robust isolated calmness of the KKT solution mapping and add insights to the existing literature on the stability of nuclear norm regularized convex optimization problems.

Face Hallucination with Weighted Nuclear Norm Constraint

Face hallucination via patch-pairs leaning based methods has been wildly used in the past several years. Some position-patch based face hallucination methods have been proposed to improve the representation power of image patch and obtain the optimal regressive weighted vector. The rationale behind the position-patch based face hallucination is the fact that human face is always highly structured and consequently positioned and it plays an increasingly important role in the reconstruction. However, in the existing position-patch based methods,the probe image patch is usually represented as a linear combination of the corresponding patches of some training images, and the reconstruction residual is usually measured using the vector norm such as 1-norm and 2-norm.Since the vector norms neglect two-dimensional structures inside the residual, the final reconstruction performance is not very satisfactory. To cope with this problem, we present a weighted nuclear-norm constrained sparse coding(WNCSC) model for position-patch based face hallucination. In addition, an efficient algorithm for the WNCSC is developed using the alternating direction method of multipliers(ADMM) and the method of augmented Lagrange multipliers(ALM). The advantages of the proposed model are twofold: in order to fully make use of low-rank structure information of the reconstruction residual, the weighted nuclear norm is applied to measure the residual matrix, which is able to alleviate the bias between input patches and training data, and it is more robust than the Euclidean distance(2-norm); the more flexible selection method for rank components can determine the optimal combination weights and adaptively choose the relevant and nearest hallucinated neighbors. Finally, experimental results prove that the proposed method outperforms the related state-of-the-art methods in both quantitative and visual comparisons.

Biquadratic tensors,biquadratic decompositions,and norms of biquadratic tensors

Biquadratic tensors play a central role in many areas of science.Examples include elastic tensor and Eshelby tensor in solid mechanics,and Riemannian curvature tensor in relativity theory.The singular values and spectral norm of a general third order tensor are the square roots of the M-eigenvalues and spectral norm of a biquadratic tensor,respectively.The tensor product operation is closed for biquadratic tensors.All of these motivate us to study biquadratic tensors,biquadratic decomposition,and norms of biquadratic tensors.We show that the spectral norm and nuclear norm for a biquadratic tensor may be computed by using its biquadratic structure.Then,either the number of variables is reduced,or the feasible region can be reduced.We show constructively that for a biquadratic tensor,a biquadratic rank-one decomposition always exists,and show that the biquadratic rank of a biquadratic tensor is preserved under an independent biquadratic Tucker decomposition.We present a lower bound and an upper bound of the nuclear norm of a biquadratic tensor.Finally,we define invertible biquadratic tensors,and present a lower bound for the product of the nuclear norms of an invertible biquadratic tensor and its inverse,and a lower bound for the product of the nuclear norm of an invertible biquadratic tensor,and the spectral norm of its inverse.

IMPULSE NOISE REMOVAL BY L1 WEIGHTED NUCLEAR NORM MINIMIZATION

In recent years,the nuclear norm minimization(NNM)as a convex relaxation of the rank minimization has attracted great research interest.By assigning different weights to singular values,the weighted nuclear norm minimization(WNNM)has been utilized in many applications.However,most of the work on WNNM is combined with the l 2-data-fidelity term,which is under additive Gaussian noise assumption.In this paper,we introduce the L1-WNNM model,which incorporates the l 1-data-fidelity term and the regularization from WNNM.We apply the alternating direction method of multipliers(ADMM)to solve the non-convex minimization problem in this model.We exploit the low rank prior on the patch matrices extracted based on the image non-local self-similarity and apply the L1-WNNM model on patch matrices to restore the image corrupted by impulse noise.Numerical results show that our method can effectively remove impulse noise.

Stable Recovery of Low Rank Matrices From Nuclear Norm Minimization

Low rank matrix recovery is a new topic drawing the attention of many researchers which addresses the problem of recovering an unknown low rank matrix from few linear measurements. The matrix Dantzig selector and the matrix Lasso are two important algorithms based on nuclear norm minimization. In this paper, we first prove some decay properties of restricted isometry constants, then we discuss the recovery errors of these two algorithms and give a new bound of restricted isometry constant to guarantee stable recovery, which improves the results of [11].

Robust Principal Component Analysis via Truncated Nuclear Norm Minimization

Robust principal component analysis(PCA) is widely used in many applications, such as image processing, data mining and bioinformatics. The existing methods for solving the robust PCA are mostly based on nuclear norm minimization. Those methods simultaneously minimize all the singular values, and thus the rank cannot be well approximated in practice. We extend the idea of truncated nuclear norm regularization(TNNR) to the robust PCA and consider truncated nuclear norm minimization(TNNM) instead of nuclear norm minimization(NNM). This method only minimizes the smallest N-r singular values to preserve the low-rank components, where N is the number of singular values and r is the matrix rank. Moreover, we propose an effective way to determine r via the shrinkage operator. Then we develop an effective iterative algorithm based on the alternating direction method to solve this optimization problem. Experimental results demonstrate the efficiency and accuracy of the TNNM method. Moreover, this method is much more robust in terms of the rank of the reconstructed matrix and the sparsity of the error.

The bounds of restricted isometry constants for low rank matrices recovery

This paper discusses conditions under which the solution of linear system with minimal Schatten-p norm, 0 〈 p ≤ 1, is also the lowest-rank solution of this linear system. To study this problem, an important tool is the restricted isometry constant （RIC）. Some papers provided the upper bounds of RIC to guarantee that the nuclear-norm minimization stably recovers a low-rank matrix. For example, Fazel improved the upper bounds to δ4Ar 〈 0.558 and δ3rA 〈 0.4721, respectively. Recently, the upper bounds of RIC can be improved to δ2rA 〈 0.307. In fact, by using some methods, the upper bounds of RIC can be improved to δ2tA 〈 0.4931 and δrA 〈 0.309. In this paper, we focus on the lower bounds of RIC, we show that there exists linear maps A with δ2rA 〉1√2 or δrA 〉 1/3 for which nuclear norm recovery fail on some matrix with rank at most r. These results indicate that there is only a little limited room for improving the upper bounds for δ2rA and δrA.Furthermore, we also discuss the upper bound of restricted isometry constant associated with linear maps A for Schatten p （0 〈 p 〈 1） quasi norm minimization problem.

Multiple Change Points Detection in High-Dimensional Multivariate Regression

This paper considers the problem of detecting structural changes in a high-dimensional regression setting. The structural parameters are subject to abrupt changes of unknown magnitudes at unknown locations. The authors propose a new procedure that minimizes a penalized least-squares loss function via a dynamic programming algorithm for estimating the locations of change points. To alleviate the computational burden, the authors adopt a prescreening procedure by eliminating a large number of irrelevant points before implementing estimation procedure. The number of change points is determined via Schwarz’s information criterion. Under mild assumptions, the authors establish the consistency of the proposed estimators, and further provide error bounds for estimated parameters which achieve almost-optimal rate. Simulation studies show that the proposed method performs reasonably well in terms of estimation accuracy, and a real data example is used for illustration.

Adaptive sparse and dense hybrid representation with nonconvex optimization

Sparse representation has been widely used in signal processing,pattern recognition and computer vision etc.Excellent achievements have been made in both theoretical researches and practical applications.However,there are two limitations on the application of classification.One is that sufficient training samples are required for each class,and the other is that samples should be uncorrupted.In order to alleviate above problems,a sparse and dense hybrid representation(SDR)framework has been proposed,where the training dictionary is decomposed into a class-specific dictionary and a non-class-specific dictionary.SDR putsℓ1 constraint on the coefficients of class-specific dictionary.Nevertheless,it over-emphasizes the sparsity and overlooks the correlation information in class-specific dictionary,which may lead to poor classification results.To overcome this disadvantage,an adaptive sparse and dense hybrid representation with non-convex optimization(ASDR-NO)is proposed in this paper.The trace norm is adopted in class-specific dictionary,which is different from general approaches.By doing so,the dictionary structure becomes adaptive and the representation ability of the dictionary will be improved.Meanwhile,a non-convex surrogate is used to approximate the rank function in dictionary decomposition in order to avoid a suboptimal solution of the original rank minimization,which can be solved by iteratively reweighted nuclear norm(IRNN)algorithm.Extensive experiments conducted on benchmark data sets have verified the effectiveness and advancement of the proposed algorithm compared with the state-of-the-art sparse representation methods.

Improved image denoising via RAISR with fewer filters

In recent years,accurate Gaussian noise removal has attracted considerable attention for mobile applications,as in smart phones.Accurate conventional denoising methods have the potential ability to improve denoising performance with no additional time.Therefore,we propose a rapid post-processing method for Gaussian noise removal in this paper.Block matching and 3D filtering and weighted nuclear norm minimization are utilized to suppress noise.Although these nonlocal image denoising methods have quantitatively high performance,some fine image details are lacking due to the loss of high frequency information.To tackle this problem,an improvement to the pioneering RAISR approach(rapid and accurate image super-resolution),is applied to rapidly post-process the denoised image.It gives performance comparable to state-of-the-art super-resolution techniques at low computational cost,preserving important image structures well.Our modification is to reduce the hash classes for the patches extracted from the denoised image and the pixels from the ground truth to 18 filters by two improvements:geometric conversion and reduction of the strength classes.In addition,following RAISR,the census transform is exploited by blending the image processed by noise removal methods with the filtered one to achieve artifact-free results.Experimental results demonstrate that higher quality and more pleasant visual results can be achieved than by other methods,efficiently and with low memory requirements.

An Image Denoising Method Based on Group Sparsity and Low Rank

In this paper,we propose an image denoising method combining the priors of non-local self similarity(NSS),low rank and group sparsity.In the proposed scheme,the image is decomposed into overlapping patches,and then these patches are classified by the K-means clustering.Patches in each cluster are stacked into a matrix and then are decomposed into low frequency component and high frequency component through 2-D wavelet transform.Intuitively,the low frequency component should be a low rank matrix.We show that the high frequency component can be recovered by weighted mixed norm minimization which is also known as group sparse model.Then we propose an image denoising model using nuclear norm and weighted mixed norm as regularizers to enforce the priors on the low and high frequency.The proposed model can be solved efficiently in the framework of alternating direction multiplier method(ADMM)algorithm.Several experiments are carried out to verify the performance of the proposed model.

Low Rank and Total Variation Based Two-Phase Method for Image Deblurring with Salt-and-Pepper Impulse Noise

Although there are many effective methods for removing impulse noise in image restoration,there is still much room for improvement.In this paper,we propose a new two-phase method for solving such a problem,which combines the nuclear norm and the total variation regularization with box constraint.The popular alternating direction method of multipliers and the proximal alternating direction method of multipliers are employed to solve this problem.Compared with other algorithms,the obtained algorithm has an explicit solution at each step.Numerical experiments demonstrate that the proposed method performs better than the stateof-the-art methods in terms of both subjective and objective evaluations.

A Patch-Based Low-Rank Minimization Approach for Speckle Noise Reduction in Ultrasound Images

Ultrasound is a low-cost,non-invasive and real-time imaging modality that has proved popular for many medical applications.Unfortunately,the acquired ultrasound images are often corrupted by speckle noise from scatterers smaller than ultrasound beam wavelength.The signal-dependent speckle noise makes visual observation difficult.In this paper,we propose a patch-based low-rank approach for reducing the speckle noise in ultrasound images.After constructing the patch group of the ultrasound images by the block-matching scheme,we establish a variational model using the weighted nuclear norm as a regularizer for the patch group.The alternating direction method of multipliers(ADMM)is applied for solving the established nonconvex model.We return all the approximate patches to their original locations and get the final restored ultrasound images.Experimental results are given to demonstrate that the proposed method outperforms some existing state-of-the-art methods in terms of visual quality and quantitative measures.

Degrees of freedom in low rank matrix estimation

The objective of this paper is to quantify the complexity of rank and nuclear norm constrained methods for low rank matrix estimation problems. Specifically, we derive analytic forms of the degrees of freedom for these types of estimators in several common settings. These results provide efficient ways of comparing different estimators and eliciting tuning parameters. Moreover, our analyses reveal new insights on the behavior of these low rank matrix estimators. These observations are of great theoretical and practical importance. In particular, they suggest that, contrary to conventional wisdom, for rank constrained estimators the total number of free parameters underestimates the degrees of freedom, whereas for nuclear norm penalization, it overestimates the degrees of freedom. In addition, when using most model selection criteria to choose the tuning parameter for nuclear norm penalization, it oftentimes suffices to entertain a finite number of candidates as opposed to a continuum of choices. Numerical examples are also presented to illustrate the practical implications of our results.