一种改进的基于记忆的自适应汉语语言模型

An Improved Cache-based Adaptive Chinese Language Model

基于记忆的自适应语言模型虽然在一定程度上增强了语言模型对不同领域的适应性 ,但其假设过于简单 ,即认为一个在文章的前面部分出现过的词往往会在后面重复出现。通过对一些文本的观察分析 ,我们认为作者在书写文章的时候 ,除了常常使用前文中出现过的词汇外 ,为了避免用词单调 ,还会在行文过程中使用前文出现过词汇的近义词或者同义词。另外 ,一篇文章总是围绕某个主题展开 ,所以在文章中出现的许多词汇往往在语义上有很大的相关性。我们对基于记忆的语言模型进行了扩展 ,利用汉语义类词典 ,将与缓存中所保留词汇语义上相近或者相关的词汇也引入缓存。实验表明这种改进在很大程度上提高了原有模型的性能 ,与n元语言模型相比困惑度下降了 4 0 1% ,有效地增强了语言模型的自适应性。

Even if n-grams language models were proved to be very powerful and robust in various tasks, they have a certain handicap that the dependency is limited to very short local context because of the Markov assumption. Though cache-based language models adapt to cross-domain environment very well, the hypothesis behind this language model is too simple. It assumes that a word that has been used often reappears in the same document. We extend this model by introducing the Chinese concept lexicon into it. The cache of the extended language model contains not only the words occurred recently but also the semantically related words. Experiments have shown that the performance of the adaptive model has been improved greatly and the perplexity has decreased almost 40.1% compared with n-gram language model.