无监督的动态分词方法

Unsupervised Dynamic Word Segmentation

介绍了一种变长汉语语料自动分词方法，这种方法以信息理论中极限熵的概念为基础，运用汉字字串间最大似然度的概念，对汉语语料进行自动分词．讨论了这些方法的局限性，并列出了一些试验结果．

A variable distance automatic word segmentation method to Chinese corpus is presented.It is based on the concept of limiting entropy in information theory, and utilizes the maximum likelihood between the strings of Chinese characters to do automatic Chinese word segmentation.A method of establishing unsupervised dynamic word segmentation dictionary is specially studied.The limitations of these methods are described.Some experimental results are also covered.