迭代重加权最小二乘支持向量机快速算法研究

Study on the Fast Training Algorithm of Iteratively Re-weighted Least Squares Support Vector Machine

迭代重加权(Iteratively Reweighted)方法是提高最小二乘支持向量机(LS-SVM)稳健性的重要手段,但由于涉及到多次加权和重复训练,该方法需要大量运算,无法广泛应用。通过数值推导,获得了求解迭代重加权最小二乘支持向量机(IRLS-SVM)的快速算法,大幅度减少了其运算复杂度。引入了3种经典的加权函数,并在多个仿真数据集和实际数据集上进行实验,证实了IRLS-SVM能获得相当稳健的学习结果,所提出的快速算法也确实能够大幅度减少训练时间。实验结果同时表明,在快速训练算法的框架下,3种不同的权重函数可能要求不同的训练时间。

Iteratively reweighted method is an important approach to improve the robustness of least squares support vector machine（LS-SVM）. However, the reweighting and retraining procedure demands a lot of computational time, which makes it impossible for practical applications. In this paper, the iteratively reweighted least squares support vector machine （IRLS-SVM） was studied. An improved training algorithm of IRLS-SVM was proposed. It is based on novel numerical method, and can effectively reduce the computational complexity of IRLS-SVM. Three different weight functions were implemented in the IRLS-SVM. Experiments on simulated instances and real-world datasets demonstrate the validity of this algorithm. Meanwhile, the results reveal that different weight function may require different computational time for the fast training algorithm of IRLS-SVM.