基于Hadoop的地理国情监测高性能计算分析研究

Research on the High Performance Computing and Analysis of Geographical Conditions Monitoring Based on Hadoop

随着常态化地理国情监测在我国的全面开展,地理国情时空数据规模不断增大,使得对地理国情数据综合分析和信息挖掘的要求越来越高。本文针对传统空间数据存储和分析平台存在的性能瓶颈问题,提出了一种基于Hadoop的地理国情高性能计算框架,论述了通过服务封装实现分布式并行计算能力与成熟平台地理空间分析能力的无缝聚合实现最优化地理国情分析计算的理念,并以地理国情地表覆盖面积统计为例,探讨了分布式计算技术在地理国情海量数据分析方面的适用性。结果表明,随着数据量的增加,分布式计算集群的性能优势逐渐凸显,比传统集中式空间数据库更适合于海量数据的分析计算,而集群的良好横向拓展性,也为大规模地理国情空间数据分析提供了可行解决方案,研究结果将为进一步构建开放性地理国情分析服务平台奠定基础。

With the normalization of the geographical conditions monitoring in China,the amount of geographical conditions spatio-temporal data is continuously increasing,which makes the demand for comprehensive analysis and information mining of geographical conditions data become higher and higher.To resolve the bottleneck problems of traditional data storage and analysis platform,a high performance computing and analysis framework of geographical conditions is proposed.Then,the idea of seamlessly combining distributed parallel computing technology and mature geographical spatial analysis technology with service encapsulates to achieve optimized calculation is discussed.Finally,a case study of geographical conditions surface area statistic is carried out to confirm the capability of distributed computing technology in huge amounts spatial data analysis.The result shows that with the increasing of data amount the performance advantage of distributed computing cluster is gradually highlighted.The cluster shows the good performance of horizontal sacaling up,which will provide a feasible solution for largescale spatial data analysis in geographical conditions monitoring.The research results confirme a good foundation for further construction of the geographical conditions analysis and service for open platform.