一种新的支持向量机主动学习策略

A novel support vector machine active learning strategy

本文提出一种新的支持向量机(support vector machine,SVM)主动学习策略,称为Dix_SVMactive.通过定义新的数据置信度度量来挑选最有价值样本进行人工标注,并在每次迭代中对训练集的平衡度进行调整,以获得更好的泛化能力.在UCI标准数据集上的测试结果表明,与基于随机选样的SVMactive和传统SVMactive(Tong SVMactive)方法相比,本文算法不仅可以提高分类精度,而且能减少人工标注的工作量.

This paper proposes a new strategy of active learning for support vector machine （SVM）, which is called Dix-SVMactive. Generally, the shorter the distance between the sample and the hyperplane, the more uncertainty and more information the sample contains, and thus it is of more value. Active learning is an iterative process, so the convergence speed should also be considered. In this paper, by defining a new confidence measure parameter about samples, the most valuable samples will be selected to be marked artificially. The confidence of a given unlabeled sample, which can be regarded as the value of the sample, is defined as the ratio of the mean value of the distance between the presented sample and the labeled samples to the distance between the presented sample and the hyperplane. While, the mean value of the distance between the presented sample and the labeled samples can measure the redundancy rate of the given sample to labeled samples, and the distance between the presented sample and the hyperplane can express the uncertainty of the sample. In general, the bigger the former and the smaller the latter, the bigger is the confidence of the sample. Additionally, the set of labeled sample obtained after each loop may be unbalanced, which means the hyperplane may be a little far away from one kind of samples and more close from another kind of samples. In this situation, according to the proposed approach to select samples, the number of samples close to the hyperplane will be more than that far from the hyperplane, and this may be lead to bad generalization performance. To avoid the unbalance of dataset, after each loop the proposed algorithm will test the balance degree of the dataset, which is the ratio of the number of minority samples to that of majority ones. When the ratio is not greater than a given threshold e, the dataset will be regarded as unbalanced. At this time, some samples belonging to the majority samples will be deleted by some strategy like clustering to make numbers of two classes samples be equal. During each iterative step, the balance degree of the selected dataset will be adjusted so as to obtain good generalization ability. Summarily, the confidence of each sample is computed firstly, and then the first a few samples will be added into the training dataset according to the confidence in descend sort. At last, the balance of the training dataset in each loop will be adjusted. The experiment results on University of California Irvine benchmatk datasets demonstrate that the proposed approach can not only improve the classification precision, but also reduce the workload of marking samples artificially compared to some common used approaches, e. g. , the SVMactive, which is based on the random sample, and the Tong SVMactive approach.