基于马氏距离的孪生多分类支持向量机

Mahalanobis Distance-based Twin Multi-class Classification Support Vector Machine

孪生支持向量机(TWSVM)的研究是近来机器学习领域的一个热点。TWSVM具有分类精度高、训练速度快等优点,但训练时没有充分利用样本的统计信息。作为TWSVM的改进算法,基于马氏距离的孪生支持向量机(TMSVM)在分类过程中考虑了各类样本的协方差信息,在许多实际问题中有着很好的应用效果。然而TMSVM的训练速度有待提高,并且仅适用于二分类问题。针对这两个问题,将最小二乘思想引入TMSVM,用等式约束取代TMSVM中的不等式约束,将二次规划问题的求解简化为求解两个线性方程组,得到基于马氏距离的最小二乘孪生支持向量机(LSTMSVM),并结合有向无环图策略(DAG)设计出基于马氏距离的最小二乘孪生多分类支持向量机。为了减少DAG结构的误差累积,构造了基于马氏距离的类间可分性度量。人工数据集和UCI数据集上的实验均表明,所提算法不仅有效,而且相对于传统多分类SVM,其分类性能有明显提高。

Twin support vector machine （TWSVM） is a recent hot spot in the field of machine learning.TWSVM achieves fast training speed and good performance for data classification.However,it does not take full advantage of the statistical information in training samples.As an improved algorithm of TWSVM,mahalanobis distance-based twin support vector machine （TMSVM） takes the covariance of each class of data into consideration and is suitable for many real problems.However,the learning speed of TMSVW remains to be improved.Moreover,the approach only deals with binary classification problems.To solve these problems,this paper formulated a least squares version of TMSVM by considering equality type constraints instead of inequalities of TMSVM,which makes the solution follow from solving a set of linear equations instead of quadratic programming,and extended the binary classier into a new multiclass classification algorithm with directed acyclie graph （DAG）.In order to reduce error accumulation in DAG structure,a mahalanobis distance-based distance measure was designed as the class separability criterion.The experimental results on toy dataset and UCI datasets show that the proposed algorithm is efficient and has better classification accuracy than traditional multi-class SVM.