汉维语短语搭配的识别和对齐

CHINESE-UYGHUR PHRASES COLLOCATION AND ALIGNMENT

提出一种简单实用的汉维语短语搭配的抽取方法。该方法不需要汉语分词、词性标注等预处理工作,根据语料中汉语字和维语单词的共现信息,避免语料中个别词汇数目极少而共现信息值较大出现噪音,采用t检验消除,相对于利用分词和词性标注等技术的抽取方法,该算法简单且时间效率较高。实验结果表明,该方法利用较小规模的语料库也能达到较好的短语搭配抽取效果。

This dissertation puts forward a simple and practical extraction method for Chinese and Uyghur phrases collocation.The method does not require Chinese word segmentation,POS tagging and other pre-processing works,according to co-occurrence information of Chinese characters and Uyghur words in the corpus,it avoids the existence of the noise caused by sparseness of exceptional words but with quite big co-occurrence information value in the corpus and uses t-test to eliminate the noise.Compared with traditional extraction methods based on word segmentation and POS tagging technologies,the algorithm is simple and time efficient.Experimental results show that the method can achieve preferable effect of phrases collocation extraction even using smaller corpus library.