基于Scrapy的招聘信息爬虫设计与实现

随着互联网发布的各种数据在急剧增长,人们正常进入网站精确获取信息的速度较慢而且信息量较小,按照一定的规则编写的计算机指令——网络爬虫应运而生,它可以在较短的时间内从网络上自动抓取大量数据信息。该文研究的是基于Scrapy框架的一种招聘信息主题网络爬虫的设计与实现。主题网络爬虫只搜索与主题信息有关的资源数据,用来服务于高校毕业生群体,作为就业前收集招聘信息的辅助手段。

With the rapid growth of various data released by the Internet,people normally enter the website to obtain accurately information is Comparatively slower and the amount of information is less.The computer instructions written according to certain rules——web crawlers came into being,it can be shorter Automatically grab large amounts of data from the network during the time.This paper studies the design and implementation of a recruitment information topic web crawler based on the Scrapy framework.The theme web crawler only searches for resource data related to the topic information,which is used to serve the college graduates as an auxiliary means of collecting recruitment information before employment.