基于流式计算引擎的时空查询中间件设计与实现

Design and implementation of spatio-temporal query middleware based on streaming computing engine

文章扩展了Spark-Streaming的功能,使之能够支持地学流式计算场景下的时空查询。首先,设计并实现了一个中间件,该中间件提供了对空间对象、空间索引、空间分区的支持。其次,基于这个中间件,在Spark-Streaming上实现了一组经典的时空查询算法,这些算法包括空间最近邻域搜索(Spatial K Nearest Neighbor Query,KNN Query)、区域查询(Range Query,RQ)。最后,在一个大规模真实数据集和一个8节点集群上验证了这些算法,评估了流式计算场景下的时延、吞吐等性能指标。

In this paper, we extend Spark-streaming to support geographic computation in streaming scenario. Firstly,we design and implement a middleware that supports spatial objects, spatial indexing, and spatial partitioning.Secondly, based on this middleware, we implement two classical spatio-temporal query algorithms on SparkStreaming. These are Spatial K Nearest Neighbor Query(KNN), and Region Query(RQ), respectively. Finally, we validate these algorithms on a large-scale real dataset in an eight node cluster, and evaluate the performance metrics such as latency and throughput in a streaming computing scenario.