基于词典、规则的斯拉夫蒙古文词切分系统的研究

Research on Slavic Mongolian Word Segmentation Based on Dictionary and Rule

斯拉夫蒙古文是蒙古国现行的文字,又称为西里尔蒙古文或新蒙古文。蒙古文词干和词缀包含着大量信息,斯拉夫蒙古文词切分是斯拉夫蒙古文信息处理众多后续工作的基础。该文尝试了将词典和规则结合的方法对斯拉夫蒙古文进行词切分。首先预处理部分蒙古文词,然后基于词典切分高频和部分不符合规则的词。最后对剩余的词,用切分规则生成多个候选的词切分方案,然后在这些方案中选出最优方案。通过两种方法的有机结合,发挥各自的优点,得到了性能较好的斯拉夫蒙古文词切分系统。

Slavic Mongolian is the daily language in Mongolia, which is also known as Cyrillic Mongolian or new Mongolian. This paper explores the Slavic Mongolian word segmentation by combining the dictionary with rules. We first preprocess with the dictionary for the words of high-frequency or not consistent with rulesm then deal with the rest words with rules to generate n-best candidates for final decision We combine the two different methods, taking bothadvantages and achieving excellent performance in the Slavic Mongolian word segmentation.