基于Spark的矢量数据叠加赋值方法研究与实现

Research and Implementation of Vector Data Value Written Using Overlay Analysis Based on Spark

随着GIS数据获取与处理技术的迅速发展,以土地利用为代表的矢量空间数据规模不断膨胀,大量生产应用对图层间矢量数据叠加赋值操作性能提出了更高要求.本文提出了基于Apache Spark技术的矢量数据叠加赋值方法,通过扩展Spark技术的弹性分布式数据集,使其提高对于GIS空间数据的表达能力,通过空间索引的构建使得叠加计算可以在Spark集群各节点上分布式高效运行.通过十万、百万、千万3种量级的数据进行实验,结果表明,相比传统算法,基于Spark技术的矢量数据叠加赋值方法有30%—90%的性能提升.

With the rapid development of GIS data acquisition and processing technology, the scale of spatial vector data represented by land use is expanding. Massive production applications put forward higher requirements for the performance of vector data overlay a-nalysis among layers. In this paper, we proposed a vector data overlay value written method based on Apache Spark. By extending the resilient distributed datasets of Spark, the RDD representation ability of spatial data is added. The spatial indexing makes the spatial calculation on the nodes of the spark cluster distributed and efficient. Experimental results show that the vector data overlay value writ-ten method based on Spark has 30% -90% performance improvement compared with traditional algorithms.