基于分布式计算的微博敏感信息挖掘系统

A Mining System for Microblog Sensitive Information based on Distributed Computing

在中国网络舆情的传播当中,微博是最有影响力的媒介之一,对微博舆情进行有效监控刻不容缓。文章设计的系统以微博舆情为研究目标,利用Mongo DB搭建了分布式计算平台,建立了敏感事件话题语料库,引入PageRank算法处理微博社交关系得到微博用户的影响力,并将高影响力微博用户发布的微博与语料库中的关键词精确匹配,进而得到微博敏感人士,最终以响应式Web界面的形式将结果呈现给用户以便其及时查看、处理。系统首次在此类系统中使用基于人(即以微博用户)为监控对象的概念。与现有部分系统相比,该系统具有高时效性、高准确率的优势。

Among the mediums of dissemination of China＇s public opinion, Microblog is one of the most influential one, and it is urgent and necessary to monitor the public opinion on Microblog effectively. The Microblog sensitive information mining system which takes the public opinion on Microblog as the research background, builds a distributed computing platform by using MongoDB. To calculate the influence of Microblog user, the system makes use of the PageRank algorithm analyzing the social relationship of Microblog users. After mapping the Microblog content of high-impact Microblog users to the keywords in the corpus of the system, sensitive users are filtered. Ultimately, all of these results are shown to the Microblog administrator in the form of responsive web interface, which makes it convenient to view and process the sensitive information. A new concept provided by the system is to monitor sensitive information based on one particular Microblog user. The system also has an advantage over other similar systems in efficiency and accuracy.