支持向量机与最小二乘法的关系研究

Relation between a support vector machine and the least square method

研究了支持向量机 (SVM)在二次损失函数下的优化问题解的形式 ,并与普通的最小二乘 (L S)估计问题进行了比较 ,得到了几乎完全一致的优化问题形式。由于 SVM在二次损失函数下的优化问题对应于一个欠定问题 ,该问题在最小二乘估计中有最小范数解。如果 SVM的参数选择合适 ,从理论上可以证明采用二次损失函数的 SVM函数拟合问题实际为约束最小二乘估计问题 ,并且该问题的解对应于最小范数最小二乘解。由于最小化范数解实际是 SVM在取某些参数时的一个特例 ,如果能够自动调整这些参数 ,则得到一类最小化范数解。由此提出了采用 SVM解决最小二乘法问题的思想 ,由于 SVM的优点 。

This paper compares the solution of a support vector machine (SVM) using a quadratic cost function with the least squares method which has a form similar to the SVM. SVM regression with a quadratic loss function is shown to be equivalent to the least squares estimate, with the SVM solution corresponding to the minimal norm least square estimate. Therefore, a minimum norm LS solution can be obtained with SVM with the proper choice of the parameters. This paper presents such a scheme for solving LS problems with the SVM technique, which is better for solving many practical problems due to the good features of SVM.