A systematic review of structured sparse learning

稀疏学习由于其简约特性和计算优势而获得了越来越多的关注,在具有稀疏性的条件下,许多计算问题可以在实践中得到有效的处理。而结构化稀疏学习则进一步将结构信息进行编码,在多个研究领域取得成功。随着各类型结构的发现,人们相继提出了各种结构化正则函数。这些正则函数通过利用特定的结构信息极大提高了稀疏学习算法的性能。在本文中,我们从想法、形式化、算法和应用等方面系统的回顾了结构化稀疏学习。我们将这些算法置于最小化损失函数和惩罚函数的统一框架中,总结了算法的开源软件实现,并比较了典型优化算法解决结构化稀疏学习问题时的计算复杂度。在实验中,我们给出了无监督学习在结构化信号恢复和层次化图像重建中的应用,以及具有图结构引导的逻辑回归的在监督学习中的应用。

High-dimensional data arising from diverse scientific research fields and industrial development have led to increased interest in sparse learning due to model parsimony and computational advantage. With the assumption of sparsity, many computational problems can be handled efficiently in practice. Structured sparse learning encodes the structural information of the variables and has been quite successful in numerous research fields. With various types of structures discovered, sorts of structured regularizations have been proposed. These regularizations have greatly improved the efficacy of sparse learning algorithms through the use of specific structural information. In this article, we present a systematic review of structured sparse learning including ideas, formulations, algorithms, and applications. We present these algorithms in the unified framework of minimizing the sum of loss and penalty functions, summarize publicly accessible software implementations, and compare the computational complexity of typical optimization methods to solve structured sparse learning problems. In experiments, we present applications in unsupervised learning, for structured signal recovery and hierarchical image reconstruction, and in supervised learning in the context of a novel graph-guided logistic regression.