基于多策略的新浪微博大数据抓取及应用

Method of Sina microblogging big data grabbing based on multi-strategy and its application

微博数据处理属于大数据范畴,其前提是获取大量的微博数据,而由于商业利益以及安全方面的考虑,获取微博数据的途径越来越少。同时随着新浪官方API的逐步更新,数据获取的限制也日益增加。文章尝试利用现有资源,基于多策略获取机制,设计出可稳定获得新浪微博数据的挖掘方案,并给出情感分析应用实例。实验表明,所设计的挖掘方案可以根据需要获取微博上的数据,并可以应用于微博情感分析中。

Dealing with microblogging data belongs to big data areas ,and the premise is to obtain a large a-mount of microblogging data .For the commercial interests as well as security considerations ,the access to the microblogging data is more and more difficult and the Sina official open API does not support large data plug-in .In this paper ,a mining scheme ,w hich is mainly based on the access mechanism of multi-strategy and ex-isting resources ,is designed to catch data stably from Sina microblog .And an example of its application in sentiment analysis is given .The test results show that the proposed mining scheme can catch microblogging data as needed and it is suitable for the sentiment analysis of microblog .