增量式最小二乘法分类器与增量式支持向量机的对比

Incremental Least Squares Classifier vs.Incremental Proximal Support Vector Machine

在处理大规模数据时,近似支持向量机及其增量式版本(ISVM)是一种比传统支持向量机更加简单而有效的分类器.但在处理高维数据时,由于ISVM通过计算矩阵的逆来更新模型参数,这使得其计算效果有待提高.针对上述问题,本文提出了基于最小二乘法的增量式方法.该增量式方法通过对矩阵运算的恒等推导,把矩阵求逆问题转变成了除法运算,得到了简单的模型参数更新公式,从而获得了和ISVM同样的预测精度,且在处理高维数据时运行效率更高.在合成数据及图像和生物数据上的试验表明该增量式方法优于ISVM方法.

Proximal support vector machine （PSVM） and its incremental version（ ISVM） are much simple but more effective classifiers than traditional support vector machines （ SVMs）, for dealing with large volume data. Nevertheless, the computational efficiency of the ISVM for high dimensional data is still unsatisfactory, mainly because that it requires explicit matrix inversion for updating model parameter. In this paper, we propose an incremental solution which relies on least squares to get identical prediction accuracy to that of ISVM, but is more efficient for high dimensional data. More specifically, we solve incremental least squares problem by identically transforming the matrix inversion into simple division operation, and obtain simple formulas of model updating. Experimental results on synthetic and real-world （ image and biological data） confirm that the proposed incremental classifier obtains the same pre- diction accuracy as that of the ISVM, moreover, the proposed method performs much better than ISVM for high dimensional data.