计算机彝文自动分词技术的设计研究

Design and Implementation of Yi Automatic Segmentation Technique in Computer

实现彝语文自动分词是计算机彝文信息处理中一项不可缺少的基础性工作,计算机彝文信息处理只要涉及到信息检索、机器翻译、语法分析、语义分析等方面的应用,就都需要以词为基本的处理单位.论文以彝语言的特点作为出发点,首先提出了计算机彝文分词规则与分词词表的设计思路,其次提出了实现计算机彝文自动分词技术的算法基础、系统结构,以及实现流程,而且进行了抽样测试,其分词的速度和准确率都比较高.论文最后根据彝语言的特点对实现计算机彝文自动分词的难点进行了分析.

The automatic word segmentation is an indispensable basic work of Yi language information processing. As long as Yi language information processing related to the retrieval, translation, syntactic a- nalysis , semantic analysis,it requires the use of word as basic unit. On this basis according to characteris- tics of Yi language,the automatic word segmentation standard and design of word vocabulary are described. The technology of automatic word segmentation is proposed, which based on established vocabulary of Yi language. The technology includes algorithm selection, system architecture,and implementation process. And sample tests are given, the accuracy rate and speed of word segmentation are quite satisfactory. Finally, on characteristics of Yi language and the difficulty of achieve automatic word segmentation is analyzed.