中文全文检索系统中的压缩模型和模式匹配技术

The Compression Scheme and Pattern Matching for Chinese Text

本文给出了一种适用中文全文检索系统的压缩模型 ,使传统的LZW模型能适用于大字符集语言源文本。方法的关键是通过引入切割标记控制字典多叉树的节点的无限扩大。对文件的检索直接在压缩文件上进行 ,因而可较大地提高检索效率。

We propose an efficient compression scheme for Chinese text which is based on the useful LZW method.The general purpose compression utilities is not suited for Chinese text for its large alphabet.The key technique in our scheme is“Chinese words segment signs”which could reduce the size of the tree dictionary.The retrieve of the document is processed in the compressed file directly,therefore,allowing faster search at the same time.