彝文网页文本分词平台

Analysis and Discussion of Yi Word Segmentation

在机器翻译、自动分类、搜索引擎等技术中,彝文分词具有很重要的作用,同时也是彝文信息处理至关重要的环节.本文以当前的彝文分词技术为基础,通过构建彝文词库,并用彝文网页获取平台抓取彝文网页文本,结合彝文特有的优势,从分词词库、分词算法、结构流程、系统界面和模块、实验结果等方面进行了详细的分析,最终实现彝文网页文本分词平台.最后的结果表明,本平台分词准确率较高,实用性和通用性也较好.

In the fields of machine translation, automatic classificationand search engine technology, Yi word segmentation plays a very important role, which is also a vital part in Yi language information processing. This paper is based on the current segmentation of Yi word.Through the construction of Yi thesaurus and webpages of Yi, we can grab the page texts of Yi. Combining with the advantages of Yi language, with a detailed analysis such as the thesaurus, word segmentation algorithm, flowchart and structure, system interface and modulesand the experimental results, we build the segmentation platform of Yi page text. Finally, the results show that the segmentation platform has a property of higher accuracy, practicality and versatility.