基于Scrapy的太原二手房爬虫的设计与实现

Design and Implementation of Second Hand House Crawler in Taiyuan Based on Scrapy

随着经济的快速发展,人们买房获取数据信息越来越便利,但网络的海量数据也给人们快速摘取有用信息带来了很大的困难,同样太原二手房数据量也很庞大。利用Scrapy框架快速高效的爬取太原链家二手房数据,并对数据完成处理和存储,为买房人群提供有用信息数据参考。

With the rapid development of economy,it is more and more convenient for people to obtain data information for buying a house,but the massive data on the Internet also brings great difficulties for people to quickly extract useful information.Similarly,Taiyuan has a huge amount of second-hand house data.In this paper,it uses the Scrapy framework to quickly and efficiently crawl the second-hand house data of Taiyuan Chain Home,complete the processing and storage of the data,and provide useful information and data reference for people who buy houses.