基于最大熵的维吾尔语句子边界识别模型

Uyghur Sentence Boundary Identification Model Based on Maximum Entropy

采用最大熵模型实现维吾尔语句子边界识别,该模型的训练过程不需要提供手工收集规则、词性标注及形态分析,仅使用较容易获取的单词长度和音节等特征。为确定最佳特征模板,在特征空间上组合出不同特征模板进行测试。实验结果表明,最佳特征模板具有较强的鲁棒性,召回率可达97.72%。

The Maximum Entropy（ME） model is used to detect Uyghur sentence boundary. The training procedure does not require hand-crafted rules, parl-of-speech tags, or morphological information, but uses readily available features, such as word length and syllable. To determine the best feature set, tests are performed on the different combinations of features. Experimental results show the best feature set is quite robust, and achieves recall of 97.72%.