基于组合度的汉语分词决策算法研究

Chinese word segmentation arithmetic based on the degree of combination

提出了汉字组合的组合度概念 ,讨论了组合度与组合的成词能力之间的关系 ,利用决策树的方法挖掘了组合度与分词模板的关系 .在此基础上得出了一种新的分词算法 .实验表明组合度对组合成词能力的影响远远大于组合频率的影响 .这种分词方法对汉语分词的歧义问题、人名、地名识别问题 ;新词识别问题等都有一定的作用 .

In this paper, we put forward the conception of combination degree about Chinese word; discussed the relationship between combination degree and the to_binded word capacities; proposed the method of Chinese word segmentation by means of combination degree and word segmentation template. Experiments show that this method can better solve the problem of different meaning in Chinese word segmentation, recognition of person and place name as well as new words.