超大规模栅格数据管理系统的设计与实现

Design and implementation of ultra large scale raster data management system

本文基于HDFS分布式文件系统和Spark分布式分析框架,首先,构建了超大规模栅格敎据管理系统,实现了不切片动态渲染分布式栅格地图服务,并采用移动计算到数据的策略和多级缓存机制,极大提高了动态渲染性能;然后,设计了栅格分布式分析数据模型RasterRDD,并利用Spark框架多节点分布式计算能力,大幅提升了栅格分析效率;最后,通过发布全国D0M和DEM栅格分布式地图服务,进行动态渲染和坡度分析性能验证,能够满足超大规模栅格数据的高效浏览、存储和分析。

Based on the Hadoop distributed file system(HDFS)distributed file system and Spark distributed analysis framework,an ultra large scale raster data management system is constructed,realizing the dynamic rendering of distributed raster map services without cutting tiles.The strategy of migrating calculation to data and the multi-level caching mechanism is adopted to greatly improve the dynamic rendering performance.A distributed raster analysis data model RasterRDD is designed and the Spark framework's multi-node distributed computing capability is employed to obviously improve the efficiency of raster analysis.Finally,through the publication of national DOM and DEM raster distributed map services,dynamic rendering and slope analysis performance verification are conducted,which can meet the requirement of efficient browsing,storage and analysis of ultra large scale raster data.