基于Spark的大规模图数据并行计算研究

Research on Large-Scale Graph Parallel Computing Based on Spark

随着社交网络的兴起,大规模图数据处理技术成为研究的热点,从海量的社交数据中分析数据的关系具有巨大的商业价值。Spark利用其内存计算模型和适合迭代运算的优势,为大规模图数据并行运算提供Graphx框架。以经典的Page Rank算法为例,分析Graphx框架下的Pregel迭代计算模型,总结Pregel计算模型的优势和应用场景。

With the development of social network, large-scale graph processing technology become a hot spot of research. Analyzing relationship from massive social data has great commercial value. Taking the advantages of memory-computing model and iterative computation, Spark provides Graphx for large-scale graph parallel computing framework. Analyzes the Pregel iterative computing model under Graphx in the example of classical Page Rank algorithm, summarizes the advantages and application of Pregel computing model.