基于2-gram语言模型的哈萨克文语料库校对研究

Kazakh Corpus Proofreading Research Based on 2-gram Language Model

研究了带有位置信息的2-garm音节模型,根据音节切分后前后音节的位置信息和同现概率来判断非词错误,再根据最短编辑距离和viterbi算法来提供候选词.根据2元单词模型,对输入文本以句子为单位,从判断前后2元单词的同现概率来判断真词错误,并提供候选词.

In this paper,it is studying 2-gram syllable model with position information,according to the syllable before and after position information,after syllable segmentation and together present probability to judge non-word errors,then using minimum edit distance algorithm and viterbi algorithm to offer candidate words. Dividing input texts into sentences based on 2-gram word model,so as to judge real word errors by together present probability of 2-gram words before and after,also offering candidate words.