基于Hadoop的微博舆情监控系统模型研究

Hadoop Based Public Opinion Monitoring System for Micro-blogs

【目的】针对当前的大数据环境,提出基于Hadoop的微博舆情监控系统模型,实现对海量微博信息的采集、挖掘、监控分析。【方法】分析舆情监控技术,构建舆情监控系统模型,改进相关算法,利用Hadoop搭建大数据平台,进行仿真实验,验证模型可用性。【结果】实验结果表明,模型能够很好地对海量微博数据进行监控分析,达到舆情监控的目的。【局限】Hadoop集群规模较小;没有对比多种聚类算法,未得到改进算法与其他算法的优劣。【结论】该模型可以对海量微博数据进行舆情监控分析,为决策者应对舆情危机提供科学化的信息支持。

[Objective] This paper presents a new model for public opinion monitoring system based on Hadoop to retrieve and analyze information from the micro-blog platforms. [Methods] We first surveyed the existing technology of the public opinion monitoring systems and proposed a new model with modified algorithm. Then, we built a big data analysis platform with Hadoop to examine the model＇s feasibility through experimental simulations. [Results] The proposed model can detect and retrieve public opinion data effectively. [Limitations] The Hadoop cluster was relatively small. We did not compare our model with other clustering algorithms to discuss their advantages and disadvantages. [Conclusions] The proposed model can conduct public opinion analysis with micro-blog data and provide scientific information for the policy makers to improve crisis management.