基于虚词切分的藏文分词系统的设计与实现

Design and Implementation of a Tibetan Participle Words System Based on Function Words Segmentation

藏文分词是藏文自然语言处理的基础。根据藏文虚词在藏文文本中的特殊作用以及虚词的兼类性、结合性、黏着变体性和还原特性,设计实现了一个基于藏语虚词切分的正向最大匹配的藏文分词系统,该系统对原始文本进行逐次扫描处理得到分词结果。实验表明,文章提出的藏文分词系统具有较高的切分精度和切分速度。

The Tibetan participle word is the baseline of research on the processing Tibetan natural language.According to the special role of the Tibetan function words in the Tibetan text as well as its integrating features,agglutinative variant and restore features,a Tibetan participle words system was designed based on the forward maximum matching of Tibetan function words segmentation.The original text word is carried on scanning process gradually by this system to get segmentation results.The experiments showed that the function words segmentation correct rate possess more than 90% with the system.