Python框架下基于主题的数据爬取技术研究与实现

Research and implementation of theme-based data crawling technology with Python framework

如今上网查询和购物已经成为人们的生活必需。由于在很多系统上查看商品或资源需要点击跳转多个页面,随着浏览时间的增加,经常会出现眼花缭乱的感觉。若只为用户呈现必要的数据,必将提高筛选资源的效率。文章使用Python语言结合目前流行的Spring MVC框架来爬取目标网站的数据,设计了数据爬取模块和数据展示模块,实现了基于主题的爬虫框架。通过爬取实验与结果测试,成功爬取到了目标网站的数据并展示到自己的页面上,实现了预期的目标。

Nowadays, online enquiries and shopping have become the indispensable of people＇s daily life. Because viewing goodsor resources on many systems requires clicking and jumping over multiple pages, it is often a dazzling feeling as browsing timeincreases. If only provide users with the necessary data, the efficiency of screening resources will certainly be improved.Combining with the popular Spring MVC framework, this paper uses Python language to crawl the data of the target website,designs the data crawling module and data display module, and implements the theme-based crawler framework. The crawlingexperiment and the test result show that, the data of the target website is crawled and displayed on its own page, and theexpected goal is achieved.