基于改进决策树算法的网络关键资源页面判定

Web Key Resource Page Judgment Based on Improved Decision Tree Algorithm

关键资源页面是网络信息环境中一种重要的高质量页面,是用户进行网络信息检索的主要目标.决策树算法是机器学习中应用最广的归纳推理算法之一,适用于关键资源页面的判定.然而由于Web数据均一采样的困难性,算法缺乏有足够代表性的反例进行训练.为了解决这个问题,提出一种利用训练样例的统计信息而非个体信息进行学习的改进决策树算法,并利用这种算法实现了独立用户查询的关键资源页面判定.在2003年文本信息检索会议(TextRetrievalConference,简称TREC)标准的评测条件下,基于此种改进决策树算法的大规模网络信息检索实验获得了超过基本算法40%的性能提高.这不仅提供了一种查找Web关键资源页面的有效方式,也给出了提高决策树算法性能的一个可行途径.

Key resource page is one of the most important search target pages for Web search users. Decision tree learning is one of the most widely-used and practical methods for inductive inference in machine learning. Because of the difficulty in uniform sampling of Web pages, there are not enough negative instances for training a key resource decision tree. To solve the problem, the original algorithm is partly modified to learn from global instead of individual instance information. With the same evaluation method as TREC （Text Retrieval Conference） 2003, large scale retrieval experiments based on improved decision tree algorithm achieves more than 40% improvement than the ones based on the original algorithm. It not only offers an effective way for selecting Web key resource pages, but also shows a possible way to imorove decision tree learning performances.