基于改进SVM算法的聚焦爬虫设计与实现

Design and Implementation of Focused Search Engine Based on Improved SVM Algorithm

聚焦爬虫在搜索引擎中有着至关重要的作用。为解决传统聚焦爬虫抓取特定领域的网页信息效率低下问题,在分析传统聚焦爬虫算法的基础上,提出了一种基于改进的支持向量机(SVM)的聚焦爬虫算法。该算法针对聚焦爬虫中网页特征提取问题,采用一种基于不确定性度量UM(Uncertainty Measure)值的权重特征提取方法对聚焦爬虫算法进行优化。实验结果显示,此特征提取方法大大提高了爬虫的速率,同时查全率和查准率也有提升。

Search Engine have a vital role in search engines. In order to solve the problem that the traditional Search Engine crawles the inefficient webpage information in specific fields, an improved Search Engine algorithm based on improved Support Vector Machine (SVM) is proposed based on the analysis of traditional Search Engine algorithm. The algorithm is used to optimize the Search Engine algorithm based on the weight feature extraction method based on Uncertainty Measure (UM). The experimental results show that this feature extraction method greatly improves the rate of reptiles, and the recall and precision are improved.