大规模时空图数据存储和分析的优化方法

Optimizing large-scale spatio-temporal graph data storage and analysis

时空图数据在数据量和数据更新速率两方面具有独特的特征,可以用来优化存储和查询分析。然而,现有的成熟的大数据存储和分析系统提供统一化的支持,没有考虑结合数据特征和查询特征做针对性的优化,因而无法很好地应对大规模数据的挑战,存储和分析能力都有待加强。本文利用时空图数据的数据特征,提出了针对不同类型的顶点和边的差异化存储方案;利用时空图数据的查询特征,提出了差异化的存储布局和基于此的查询执行优化方案。实验结果表明,和现有方案相比,本研究提出的优化方法能减少1.7~5.4倍的存储空间,查询性能可以提高1~4个数量级。

Spatio-temporal graph data analysis has huge political,social and economic value.Spatio-temporal graph data has unique characteristics in terms of data volume and data velocity,those characteristics can be used to optimize big data storage and analysis.However,existing solutions provide general support,and do not consider combining data characteristics and query characteristics to make optimizations.This paper analyzes the requirement of big data system for processing spatio-temporal graph data,and proposes efficient solutions to deal with the challenges of data volume,data velocity and query processing.The experimental results show that compared with the existing solutions,this study can reduce the storage space by 1.7×-5.4×,and the query performance can be improved by 1-4 orders of magnitude.