搜索引擎检索系统质量评估

On Retrieval System Evaluation of Search Engines

搜索引擎检索系统的质量评估对传统信息检索系统评估带来了新的研究问题·利用Tiangwang搜索引擎查询日志,按类别构造评估查询集,用人工判别相关性的方法对3个搜索引擎进行了检索质量评估·实验用InfoMall系统提供的历史网页服务消除不同搜索引擎搜集系统收集网页集合的差异,得到如下结论:①评测员之间的差异很大,但评估实验结果保持稳定;②使用连续型的相关度评分以及对应的评估指标比二元相关度评分及指标具有更好的区分能力;③使用50左右规模的查询集合和DCG这样的连续型评估指标可以有效进行评估实验·

Evaluation of Web search brings challenges into the traditional evaluation methods of information retrieval systems. In this paper, the query set with different user＇s information categories is constructed by analyzing the query log of Tianwang search engine. In the evaluation experiments for three popular search engines, the differences of indexed document sets are reduced by filtering the query results on the InfoMall Web archive. Experiments show that： ①Significant differences are found in voluntary assessors, but the results of evaluation keep stable, ②Continuous relevant scores and corresponding measures have better distinction capability than the binary ones, and ③Query set with size of 50 is enough for the evaluation measure DCG in the Web search evaluation.