基于SVM和k-NN结合的汉语交集型歧义切分方法

A Method of Crossing Ambiguities in Chinese Word Segmentation Based on SVM and k-NN

本文提出了基于支持向量机 (SVM)和k 近邻 (k NN)相结合的一种分类方法 ,用于解决交集型伪歧义字段。首先将交集型伪歧义字段的歧义切分过程形式化为一个分类过程并给出一种歧义字段的表示方法。求解过程是一个有教师学习过程 ,从歧义字段中挑选出一些高频伪歧义字段 ,人工将其正确切分并代入SVM训练。对于待识别歧义字段通过使用SVM和k NN相结合的分类算法即可得到切分结果。实验结果显示使用此方法可以正确处理 91 .6%的交集歧义字段 ,而且该算法具有一定的稳定性。

This paper presents an algorithm based on the combination of Support Vector Maching(SVM)and k Nearest neighbor (k NN),to deal with ambiguities in Chinese word segmentation.We regard the ambiguities segmentation as a classified problem and propose a vector representation of them.The method to find the solutions is supervised learning.After the ambiguities being selected and classified by handwork,the ambiguities with high frequency are trained by SVM.For the testhing ambiguities,we classify it based on mixed classified algorithm.The experiments show that not only the correct rate can reach 91.6%.for crossing ambiguities,but also the performance of this algorithm is of high stability.