多策略融合的中文微博数据采集方法

Chinese microblog data collecting method based on multiple hybrid strategies

在基于Cookie爬虫微博数据采集方法和基于API接口微博数据采集方法的对比分析基础上,提出了一种多策略融合的中文微博数据采集方法。设计实现了广度优先的微博数据采集算法和随机活跃用户微博数据采集算法,全面高效采集中文微博中的用户ID数据、用户个人信息数据、用户微博信息数据和微博用户关注信息数据,为微博社会网络分析提供有价值的微博信息源。真实数据集上的实验结果表明,该方法不仅具有较高的采集效率,而且还具有很好的用户覆盖面。

Comparative analyzing Cookie-based crawler with API-based microblog data collecting method, a Chinese microblog data collecting method based-on multiple hybrid strategies is proposed, the ID data of users, personal information of users, microlog information data of users and information data microblog user attention are collected, valuable microblog information source is provided for microblog social network analysis. Especially a breadth-first data collecting algorithm and a random active user data collecting algorithm are designed and implemented. Online experiments show that this multiple hybrid strategy is more effective in collection and data coverage.