一种规则与统计相结合的汉语分词方法

A Method Combining Rule-based and Statistics-based Approaches for Chinese Word Segmentation

汉语自动分词是中文信息处理领域的一项基础性课题,对现有的汉语分词方法作了简单的概述和分析,然后提出了一种新的分词方法,该方法基于一个标注好了的语料库,并且结合了规则和语料库统计两种分词方法。

Chinese automatic word segmentation is a basic task in the area of Chinese NLP.After summarizing and analyzing current techniques used in Chinese word segmentation,this paper presents a new method for word segmentation which is based on a marked corpus base.The method combines rule-based and corpus-based statistical methods.