基于分组的次数与规则剪枝相结合的语言模型压缩方法研究

Research on the Language Model Compression Methods Combining the Grouping Counts and Rule Pruning

由于庞大的训练语料,统计语言模型的大小往往会超出手持设备的存储能力。随着现阶段资源受限设备的迅速发展,语言模型的压缩研究也就显得更加重要。本文提出了一个语言模型压缩方法,即将次数剪切与规则剪枝方法相结合,并使用分组的方法保证在不减少单元数目的情况下压缩模型。文章对使用新的算法得到的语言模型与次数剪切和规则剪枝方法分别进行困惑度比较。实验结果表明,使用新方法得到的语言模型性能更好。

Currently the size of most statistical language models based on large-scale training corpus always goes beyond the storage ability of many handheld devices. With the rapid development of the limited resource devices, the research on language model compression can meet such requirements. This paper proposes a language model compression method which combined the count cutoff and the pruning method to reduce the size of the language model and uses grouping to compress this model without cell reduction. Our experimental results show that our method can achieve higher perplexity than those of other methods based on the same size.