稀疏多元逻辑回归问题优化算法研究

Research on optimization algorithm for sparse multinomial logistic regression problem

稀疏多元逻辑回归(sparse multinomial logistic regression,SMLR)因为具有在分类的同时嵌入特征选择的作用而被广泛应用于生物信息学、高光谱图像分类、图像中的多类物体识别等领域。SMLR问题最早采用迭代重加权最小二乘法(iterative reweighted least squares,IRLS)的方式进行求解。但IRLS算法在处理高维数据集或者类别数较多的数据集时具有较高的计算复杂度。为了提高SMLR的可用性,提出采用一些高级优化算法如快速迭代收缩阈值法(fast iterative shrinkage threshold method,FISTA)、快速自适应收缩阈值法(fast adaptive shrinkage threshold method,FASTA)、交替方向乘子法(alternating direction multiplier method,ADMM)等来对SMLR问题进行求解。此外,为提高SMLR的适用性,还考虑了SMLR问题的分布式优化求解。对提出的几种SMLR优化求解算法的性能在不同数据集下进行了综合比较。实验结果表明,提出的算法在求解速度和准确率指标上都优于目前最先进的基于IRLS的SMLR优化算法。

Sparse multinomial logistic regression( SMLR) is widely used in disease diagnosis,multi-class object recognition,and hyperspectral image classification,etc. The SMLR problem was first solved by iterative reweighted least squares method( IRLS),but the IRLS algorithm has higher computational complexity when dealing with high dimensional datasets or datasets with a large number of classes. In order to promote high availability,we adopted a few advanced optimization algorithms,such as fast iterative shrinkage threshold method( FISTA),fast adaptive shrinkage threshold method( FASTA),alternating direction multiplier method( ADMM),to solve the SMLR problem. In addition,in order to improve the applicability of SMLR,the distributed optimization of SMLR problem is also considered. Finally,the performance of our proposed SMLR optimization algorithm is comprehensively compared among different datasets. The experimental results show that in terms of solving speed and precision rate,our proposed algorithm outperforms the state-of-the-art SMLR algorithm based on IRLS.