藏语语料库词语分类体系及标记集研究

Research on the Word Categories and Its Annotation Scheme for Tibetan Corpus

青海师范大学藏文信息处理与机器翻译省级重点实验室已完成1 000万字的藏语语料库的加工实验,加工的主要目的是使计算机能够对藏语语料库中的藏语词语进行自动切分和自动标注。该文在对大规模藏语语料库进行自动切分和人工分析的基础上提出了一个藏语词语分类体系和标记集。根据藏语语料库和计算机自动切分和标注的实际需要,在藏语词语分类体系的构建上,采用先分虚实,再确定大类,在大类的基础上分出小类,再分出不同深度的子类。在藏语语料库加工实验中的应用表明,该分类方法和标记集是一个比较合理和实用的。

For the automatic segmentation and POS tagging, this paper proposes a Tibetan word category system and a annotation scheme after a careful analysis over a large Tibetan corpus. According to the practical demands on the Tibetan corpus, the Tibetan words are first divided into several main categories according to where they are content words or function words. Then several fine granularized sub-categories are further suggested. This framework has been proved valid for the processing of a Tibetan Corpus with 10 million characters.