偏置b对支持向量机分类问题泛化性能的影响

Influence of Bias b on Generalization Ability of SVM for Classification

Poggio指出支持向量机(Support vector machine,SVM)中偏置b项是为了保证核函数的正定性,当使用的核函数为正定核时,b就不需要存在.为了验证b对SVM分类问题泛化性能的影响,研究了无bSVM的优化问题并给出了相应的有效集求解算法.通过XOR分类问题的实验研究得出约束条件N1yiαi=0会影响SVM得到最佳分类超平面.实验中的基准数据集包括了中小数据集、大规模数据集、高维数据集和多类分类数据集,并使用高斯正定核和多项式正定核作为核函数.基于26个标准数据集的实验表明无bSVM在分类问题中的计算代价要低于SVM,泛化性能要好于SVM.参数敏感性测试表明无bSVM对代价参数变化不太敏感,这使得无bSVM能在较少的参数值对中得到最佳测试精度.

It has been pointed out by Poggio that the b term in support vector machine （SVM） is to guarantee the positive definitiveness of kernel and b is not needed if the used kernel is positive definite. To testify the role of b in the generalization ability of SVM for classification, optimization formulation of SVM without b is analyzed and the corresponding active set solution algorithm is proposed. By experiments on XOR classification problem, it can be concluded that SVM would fail to reach the optimum classification hyperplane due to the existence of constraint condition ∑N1yiαi=0. Small to medium data sets, large data sets, high-dimension data sets and mutli-class classification data sets are employed in the simulations as well as the Gaussian positive definite kernel and polynomial positive definite kernel are used. The experimental results on 26 benchmark data sets show that the computational cost of SVM without b is over that of SVM and the generalization performance is over SVM. SVM without b is less sensitive to cost parameter, and this makes SVM without b reaches the optimal testing rate with less parameters pair values.