爬虫技术在农产品价格信息获取中的应用

Application of Keptile Technology in Agricultural Products Price Information Acquisition

农业信息网站建设的发展,带来农产品市场数据的急剧增长,这一增长趋势极大地推动了农业信息网站的大规模创建和使用。因此,运用大数据及爬虫技术,在农产品市场数据中把有价值的信息挖掘出来,既是适应形势发展的迫切需要,也是推进农业信息网站建设亟待解决的重要课题。提出了基于爬虫的农产品价格爬取及存储平台。爬虫技术可以按照用户需求,实时提取各类相关网页信息,通过服务平台进行整合,令用户能够在大量的价格信息中及时地获取适用信息。

Development of agricultural information website construction had brought about a dramatic increase of agricultural product market data,and this trend of growth had greatly promoted large-scale establishment and widespread use of agricultural information websites. Therefore,use of big data and reptile technology to dig out valuable information in agricultural product market data is not only an urgent need to adapt to development of the situation,but also an important issue to be urgently solved in promoting agricultural information website construction. Crawling-based agricultural products crawling and storage platform was presented. Crawler technology could be based on user needs,real-time extraction of various types of related web page information,through service platform for integration,so that users could access a large number of price information in a timely manner applicable information.