一种基于语境的中文分词方法研究

Chinese Word Segmentation Based on Language Situation

汉语不同于英语 ,词之间没有间隔标记 .而汉语分词是文本分析的第一步 ,且存在歧义切分 ,因此分词问题成为汉语分析的首要难题 .通过中文切分过程的本质分析 ,推导并提出基于马尔可夫链的语境中文切分理论 ,进而提出一种语境中文分词方法 .该方法建立在词法和句法基础上 ,从语境角度分析歧义字段 ,提高分词准确率 .

Different from English, there are no interval marks between words in C hinese, so it is difficult for word segmentation to identify ambiguous words. Established a theory of Chinese word segmentation with language situation , and proposed a Chinese word segmentation method of language situation. The metho d improves the accuracy of word segmentation, by combining morphol ogy and syntax with language situation.