基于Python的新浪微博用户数据采集与分析

Data collection and analysis for Sina weibo user based on Python

针对微博用户添加的标签体现了其自身特点及兴趣的情况,为探索微博用户添加标签的行为及特点,首先利用Python与Web自动化工具基于广度优先策略抓取微博用户、用户关系、微博内容、微博评论等数据,并将其存储在数据库中。其次,利用Pandas分析了微博数量的分布情况、用户添加标签的行为及内容,通过使用Matplotlib将分析结果可视化,并利用k-means算法对添加标签的用户进行了聚类分析。由此通过基于用户标签的聚类分析结果可用于个性化推荐及舆情分析等方面。

The tags added to Weibo users reflect their own characteristics and interests. This paper aims to explore the characteristics of users' tagging behavior. This paper first use Python and Web automation tools to capture data such as Weibo users,user relationships,blog content and comments. The captured data is stored in the database. Secondly,Pandas is used to analyze the distribution of the number of microblogs,the behavior and content of users adding tags,visualize the analysis results by using Matplotlib,and use the k-means algorithm to cluster the users who add tags. Therefore,the results of cluster analysis based on user tags can be used for personalized recommendation and public opinion analysis.