基于农业领域的网络爬虫

Web Crawler Based on Agricultural Sector

以农业环境为背景,在搜索引擎基础而又非常重要的部分——网络爬虫的基础上,提出了改进的爬虫设计,其中采用了主题相关性判断和网页选择器筛选的方法,并且通过建立对应的农产品词库,结果表明在农业领域的搜索相关性上有大幅度的提高。

Using the agricultural environment as the background,propose the improving web crawler design based on the the search engine-based and very important part-the web crawler.It uses theme-related web pages to determine the nature of judgments and screening method selector,and the establishment of the thesaurus of agricultural products.The corresponding results show that the search in the agricultural sector on the relevance of a substantial increase.