基于藏语字性标注的词性预测研究

Tibetan POS Tagging Based on Syllable Tagging

该文选取了藏语文中小学教材的部分语料,构建了带有藏语字性标记、词边界标记和词性标记的语料库,通过比较不同的分词、标注方法,证明分词、词性标注一体化效果比分步进行的效果好,准确率、召回率和F值分别提高了0.067、0.073和0.07。但词级标注模型难以解决词边界划分的一致性和未登录词的问题。基于此,作者提出可以利用字性和字构词的规律预测合成词的词性,既可以融入语言学知识又可以减少由未登录词导致的标注错误,实验结果证明,作为词性标注的后处理模块,基于字性标注的词性预测准确率提高到了0.916,这个结果已经比分词标注一体化结果好,说明字性标注对纠正词性错误标注有明显的效果。

A Tibetan corpus is constructed and annotated for the syllable markers, the word boundary markers and the part-of-speech（POS） tags, with texts selected from Tibetan textbooks of Primary and middle school. Then an empirical study reveals that the training data with the multi level annotation can enhance the effects of POS tagging. Due to the strong relation between the POS tags of words and the tags Tibetan syllables, a method of Tibetan POS tagging by the Tibetan syllables is presented. The results of experiments show that syllable tags can correct certain errors caused in POS tagging.