关于中文分词问题数据结构模型的讨论

A discussion on data structure model for Chinese word segmentation

在分词工作常用数据结构模型的基础上,提出了字符串完全分词网络模型,讨论了该模型的基本性质,给出了其中的路径查找基本算法。该模型能反映中文分词问题自身的特点,便于分析分词问题中的统一性质。利用该模型将全切分图的生成、修改、路径查找等工作分解为统一平台上相对独立的过程,能较好地配合多种常用分词算法,简化对各种算法的研究和描述。该模型与自然语言理解后续工作使用的数据结构如句法树等也有很好的相似性。

Based on the data structure model often used in Chinese Word Segmentation, a full segmentation network mod- el was proposed. The basic properties of the model were discussed. Basic algorithms for path searching in this model was given. This model can reflect the characteristics of Chinese word segmentation, and is convenient for analysing some uni- fied properties. Based on this model, tasks such as the omni--segmentation graph generation, graph modification, and path finding, can be decomposed into relatively independent processes on an unified platform. The model can match up well with many commonly used segmentation algorithms to simplify the research and description of these algorithms. The model also has similarity with data structures often used in follow--up natural language prcessing such as syntactic tree.