摘要
针对不同的网页数据环境设计有效的爬虫方案,更加有助于爬取信息的准度和效度。文中以Python及其Scrapy框架为基础,以贝壳网为例,通过对网站的页面源码结构进行深入分析,编写了行之有效的爬虫文件来获取目标网站信息。该设计方法可以应用到不同的多级页面信息类型网站,为基于Web的数据分析提供策略采集和分析支持。
Designing effective crawling strategies for different network data environments can enhance the accuracy and validity of crawling information.Taking Bakclass.com as an example,this paper analyzes the structure of the source code of the website page based on Python and its Scrapy framework environment,and then composes effective crawler files to obtain target website information.Targeting the type of different multi-level information websites,this designing method supports network data analysis based on Web by acquiring and analyzing strategies.
作者
陈皓
周传生
CHEN Hao;ZHOU Chuan-sheng(Network and Information Center,Shenyang Normal University,Shenyang 110034,China)
出处
《电脑知识与技术》
2021年第13期3-5,共3页
Computer Knowledge and Technology
基金
2020年辽宁省一流学科教育学建设项目(SNUJYX2020040402)。
