基于并行计算的网络舆情数据分析方法研究

Research on the method of network public opinion data analysis based onparallel computing

随着各类社交平台规模的扩大,网络舆情对在校学生的影响也越来越大。而传统舆情数据分析系统仅能对少量舆情数据进行分析,无法快速、准确地处理当前海量的舆情数据。针对这一问题,文中通过对舆情数据进行特征识别,设计了一套基于Hadoop架构的网络舆情数据分析系统。在该系统中,通过将CNN与BiLSTM加以融合来实现对舆情数据的分类识别,进而解决了单一算法无法获取文本上下文含义的问题。同时,该设计还将算法部署在Hadoop并行计算集群中,从而有效增加了数据的训练速度。实验结果表明,所提方法在所有对比算法中性能最优,且显著提升了算法的迭代速度,并可有效解决传统算法无法处理海量文本数据的缺陷。

With the expansion of the scale of various social platforms,network public opinion has a greater and greater impact on students.The traditional public opinion data analysis system can only analyze a small amount of public opinion data,and can’t quickly and accurately analyze the current massive public opinion data.To solve this problem,this paper designs a network public opinion data analysis system based on Hadoop architecture by identifying the characteristics of public opinion data.In this system,CNN and BiLSTM are fused to realize the classification and recognition of public opinion data,which solves the problem that a single algorithm can’t obtain the meaning of text context.At the same time,this design also deploys the algorithm in Hadoop parallel computing cluster,which effectively increases the training speed of data.Experimental results show that the proposed algorithm has the best performance in the comparison algorithm,significantly improves the iterative speed of the algorithm,and effectively solves the defect that the traditional algorithm can’t deal with massive text data.