基于Scrapy爬虫框架下电商数据分析

E-commerce Data Analysis Based on Scrapy Crawler Framework

电子商务快速发展的过程中产生了海量数据,对海量数据的分析能够挖掘有用信息,提升电商企业核心竞争力。基于此,设计了一款基于Scrapy爬虫框架下的电商数据分析平台。采用分布式抓取海量电商数据,寻找电商消费者的消费特点。在此基础上采集顾客、商品、品牌三方面数据进行整合处理,并将处理结果可视化,展现给电商企业,为电商企业快速了解消费者的消费特点,制定科学化的营销战略提供了数据参考。

The rapid development of e-commerce has produced massive data.The analysis of massive data can mine useful information and improve the core competitiveness of e-commerce enterprises.Based on this,an e-commerce data analysis platform based on Scrapy crawler framework is designed.This paper adopts distributed capture of massive e-commerce data to find the consumption characteristics of e-commerce consumers.On this basis,the data of customers,commodities and brands are collected for integrated processing,and the processing results are visualized and displayed to e-commerce enterprises,which provides data reference for e-commerce enterprises to quickly understand the consumption characteristics of consumers and formulate scientific marketing strategies.