2005年度863信息检索评测方法研究和实施

Research on the 863 Chinese Information Retrieval Evaluation （2005）

本次863中文信息检索评测的目的是检测互联网环境下大规模数据的中文信息检索技术的研究现状和系统有效性，中文与接口技术评测组综合考虑了目前信息检索面临的难点以及中文信息检索具有的特点设计了本次信息检索评测，本文详细描述了本次评测的组织过程，包括查询条件设计，语料库情况，标准答案查找方法以及评价指标和评测软件的介绍，通过对参评队伍的结果数据进行分析并结合查询条件的类型，本文还讨论了现有检索技术的优点以及存在的不足。

The objective of 863 Chinese information retrieval evaluation is to investigate the current research status and the system validity of Chinese IR system under the circumstance of mass data of WEB. The organizers consider both the difficulties the IR technology is facing and the characteristic of Chinese IR to design the evaluation. In this paper, we describe the following main procedures of the evaluation： query designing, corpus composition, relevant document set finding and the evaluation tools. We also discuss the advantages and disadvantages the current IR technology shows in the test by analyzing the performance of all the submitted runs.