The iterative hard thresholding(IHT)algorithm is a powerful and efficient algorithm for solving l_(0)-regularized problems and inspired many applications in sparse-approximation and image-processing fields.Recently,so...The iterative hard thresholding(IHT)algorithm is a powerful and efficient algorithm for solving l_(0)-regularized problems and inspired many applications in sparse-approximation and image-processing fields.Recently,some convergence results are established for the proximal scheme of IHT,namely proximal iterative hard thresholding(PIHT)algorithm(Blumensath and Davies,in J Fourier Anal Appl 14:629–654,2008;Hu et al.,Methods 67:294–303,2015;Lu,Math Program 147:125–154,2014;Trzasko et al.,IEEE/SP 14th Workshop on Statistical Signal Processing,2007)on solving the related l_(0)-optimization problems.However,the complexity analysis for the PIHT algorithm is not well explored.In this paper,we aim to provide some complexity estimations for the PIHT sequences.In particular,we show that the complexity of the sequential iterate error is at o(1/k).Under the assumption that the objective function is composed of a quadratic convex function and l_(0)regularization,we show that the PIHT algorithm has R-linear convergence rate.Finally,we illustrate some applications of this algorithm for compressive sensing reconstruction and sparse learning and validate the estimated error bounds.展开更多
Many heuristic search methods exhibit a remarkable variability in the time required to solve some particular problem instances. Their cost distributions are often heavy-tailed. It has been demonstrated that, in most c...Many heuristic search methods exhibit a remarkable variability in the time required to solve some particular problem instances. Their cost distributions are often heavy-tailed. It has been demonstrated that, in most cases, rapid restart (RR) method can prominently suppress the heavy-tailed nature of the instances and improve computation efficiency. However, it is usually time-consuming to check whether an algorithm on a specific instance is heavy-tailed or not. Moreover, if the heavy-tailed distribution is confirmed and the RR method is relevant, an optimal RR threshold should be chosen to facilitate the RR mechanism. In this paper, an approximate approach is proposed to quickly check whether an algorithm on a specific instance is heavy-tailed or not. The method is realized by means of calculating the maximal Lyapunov exponent of its generic running trace. Then a statistical formula to estimate the optimal RR threshold is educed. The method is based on common nonparametric estimation, e.g., Kernel estimation. Two heuristic methods are selected to verify our method. The experimental results are consistent with the theoretical consideration perfectly.展开更多
针对压缩传感(Compressed sensing,CS)理论中迭代硬阈值(Iterative hard thresholding,IHT)算法迭代次数多和时间长的问题,提出基于回溯的迭代硬阈值算法(Backtracking-based iterative hard thresholding,BIHT),该算法通过加入回溯的思...针对压缩传感(Compressed sensing,CS)理论中迭代硬阈值(Iterative hard thresholding,IHT)算法迭代次数多和时间长的问题,提出基于回溯的迭代硬阈值算法(Backtracking-based iterative hard thresholding,BIHT),该算法通过加入回溯的思想,优化了IHT算法迭代支撑的选择,减少支撑被反复选择的次数.模拟实验表明,在保证重建质量的前提下,相比较于IHT和正规化迭代硬阈值(Normalized IHT,NIHT)算法,BIHT算法的重建时间降低了2个数量级.用本身稀疏的0-1随机信号的重建实验表明,若测量次数和稀疏度相同,BIHT算法的重建概率高于IHT算法.展开更多
基金supported by the National Natural Science Foundation of China(No.91330102)973 program(No.2015CB856000).
文摘The iterative hard thresholding(IHT)algorithm is a powerful and efficient algorithm for solving l_(0)-regularized problems and inspired many applications in sparse-approximation and image-processing fields.Recently,some convergence results are established for the proximal scheme of IHT,namely proximal iterative hard thresholding(PIHT)algorithm(Blumensath and Davies,in J Fourier Anal Appl 14:629–654,2008;Hu et al.,Methods 67:294–303,2015;Lu,Math Program 147:125–154,2014;Trzasko et al.,IEEE/SP 14th Workshop on Statistical Signal Processing,2007)on solving the related l_(0)-optimization problems.However,the complexity analysis for the PIHT algorithm is not well explored.In this paper,we aim to provide some complexity estimations for the PIHT sequences.In particular,we show that the complexity of the sequential iterate error is at o(1/k).Under the assumption that the objective function is composed of a quadratic convex function and l_(0)regularization,we show that the PIHT algorithm has R-linear convergence rate.Finally,we illustrate some applications of this algorithm for compressive sensing reconstruction and sparse learning and validate the estimated error bounds.
文摘Many heuristic search methods exhibit a remarkable variability in the time required to solve some particular problem instances. Their cost distributions are often heavy-tailed. It has been demonstrated that, in most cases, rapid restart (RR) method can prominently suppress the heavy-tailed nature of the instances and improve computation efficiency. However, it is usually time-consuming to check whether an algorithm on a specific instance is heavy-tailed or not. Moreover, if the heavy-tailed distribution is confirmed and the RR method is relevant, an optimal RR threshold should be chosen to facilitate the RR mechanism. In this paper, an approximate approach is proposed to quickly check whether an algorithm on a specific instance is heavy-tailed or not. The method is realized by means of calculating the maximal Lyapunov exponent of its generic running trace. Then a statistical formula to estimate the optimal RR threshold is educed. The method is based on common nonparametric estimation, e.g., Kernel estimation. Two heuristic methods are selected to verify our method. The experimental results are consistent with the theoretical consideration perfectly.
文摘针对压缩传感(Compressed sensing,CS)理论中迭代硬阈值(Iterative hard thresholding,IHT)算法迭代次数多和时间长的问题,提出基于回溯的迭代硬阈值算法(Backtracking-based iterative hard thresholding,BIHT),该算法通过加入回溯的思想,优化了IHT算法迭代支撑的选择,减少支撑被反复选择的次数.模拟实验表明,在保证重建质量的前提下,相比较于IHT和正规化迭代硬阈值(Normalized IHT,NIHT)算法,BIHT算法的重建时间降低了2个数量级.用本身稀疏的0-1随机信号的重建实验表明,若测量次数和稀疏度相同,BIHT算法的重建概率高于IHT算法.
