分布式图处理系统技术综述

Survey on Technologies of Distributed Graph Processing Systems

图作为一种基本的数据类型,是对现实世界中对象及其关联关系的一种抽象.现实中,许多科学问题都可以被模型化为图的问题,因此,对图数据进行分析非常重要.图数据分析在语义Web分析、社交网络、生物基因分析以及信息检索等领域有着广泛的应用.随着移动互联、物联网等信息技术的发展,图数据的规模处于持续增长的状态.为了能够应对大规模图数据的高效分析和计算,Google提出了Pregel分布式图处理框架.此后,学术界和工业界提出了许多基于Pregel框架的优化技术和系统实现.在充分调研和分析的基础上,首先总结出分布式图处理系统的3个优化目标;其次,从计算粒度、任务调度、通信方式、负载划分这4个维度,综述现有分布式图处理系统中的各类优化技术;最后,对该领域未来的研究内容和发展方向进行了探讨与展望.

Well recognized as a primitive data structure, graph is an abstraction of objects and their pairwise connections. There exists a wide spectrum of applications, including semantic web analysis, social network analysis, biological genetic analysis and information retrieval, which can be modeled as graphs. Therefore, it is of great importance to conduct data analysis over these applications. With the development of information technology such as mobile Internet and Internet of things, the scale of graph data is increasing continuously and rapidly. To provide fast analysis over large-scale graph data, Pregel was first proposed as a distributed graph processing framework by Google. Since then, based on Pregel framework, a variety of optimization techniques and systems have been proposed by academic and industrial communities. Through extensive investigation and analysis, this paper first establishes three optimization objectives for the state-of-the-arts solutions to build distributed graph processing systems. Subsequently, it reviews mainstream optimizing techniques for the state-of-the-arts solutions from the perspective of computation granularity, task scheduling, communication mode and load balance. Finally, the paper discusses some open research problems and possible future research directions in this field.