班智达藏文标注词典设计

Design of Tibetan Part of Speech Tagging Dictionary

语料库加工是一项庞大的语言工程,其中分词标注是最基础性的工作,而分词标注词典是标注系统的重要组成,词典设计的优劣直接关系着分词标注的速度和效率。在设计国家语委项目《班智达藏文自动标注系统》的基础上,给出了分词标注词典库的结构及词典库索引查询算法。对85万字节藏语实验语料的分词和标注,分词准确率达99%,标注准确率达97%。

Corpus processing is a complicated project of language engineering, in which the segmentation and tagging are the fundamental work. The part - of speech tagging dictionary is an exxential composition of the taggging process, relating directly to the speed and efficiency of tagging. Based on design of project ＂Ban Zhi Da Titetan Au- to-tagging System＂ supported by the National Language Committee, this paper presents the construction of tagging dictionary and its index and search algorithm. The experiment on the 850 KB corpus of Tibetan shows that word segmentation accuracy rate can reach 99% and tagging accuracy rate can reach 97% .