基于文档重要度的静态索引剪枝方法

Static Index Pruning Based on Document Importance

针对网页质量参差不齐、重要程度差别巨大的问题,提出了按照网页重要程度确定其剪枝幅度的静态索引剪枝方法,并在GOV2数据集上进行了验证.实验结果表明:这种方法体现了静态索引剪枝能极大降低存储需求、提高查询效率的优点;当剪枝后的索引大小是原始大小的13%时,P@10、P@20值能达到甚至超过使用完整索引时的结果;在相同的剪枝幅度下,P@10、P@20和MAP都明显好于以往的剪枝方法.

As the quality and importance of Web pages are both variable,this paper proposes a static index pruning method which uses the web page importance to determine the ratio of information kept for each document.The result of experiments on GOV2 dataset show that（1） the proposed method greatly reduces the storage size and speeds up the search;（2） when the pruned index takes only 13% of the original size,P@10 and P@20 reach or exceed the baseline using full index;and（3） by using the proposed method,P@10,P@20 and MAP are all better than those of the traditional method at the same pruning level.