基于词典和词频的中文分词方法

Chinese Word Segmentation Method Based on Dictionary and Frequency of the Words

汉语分词是中文信息处理的前提和基础。由于中文句子中,词与词之间没有明显的分隔标志,且存在多音多义词,因此不可避免的存在切分歧义。汉语分词已成为中文信息处理的"瓶颈"。本文通过使用带有词频的词典,采用逐词分解实现了中文自动分词,并使用词频计算、歧义消除等方法提高了分词的准确率。

Chinese word segmentation is the precondition and base of the Chinese information processing.In the Chinese sentences,it has no obvious interval mark between words.It has polyphony and multi-vocal words in Chinese sentences.So,the result of segmentation unavoidable contains ambiguous words.Chinese segmentation is the "bottleneck" of the Chinese information processing.This article uses dictionary and the frequency of the word to decompose the Chinese sentence,and realizes Chinese auto-segmentation,and uses t...