基于大数据的地质云监控平台建设与应用

Construction and application of GeoCloud monitoring platform based on big data

"地质云1.0"是我国地质调查领域第一个云端的综合性集成应用平台,于2017年11月上线服务,实现了地学数据集成、地质业务集成、产品应用集成的服务效果.为了保证"地质云"能够稳定健康地运行,持续地为用户提供高质量的服务,需要对"地质云"中近400台各种类型的服务器、上万个软硬件指标进行实时监测.本文透彻分析了地质云中软硬件资源的相互关系,提出并确定了地质云中各种资源的量化监测参数,首次提出了在云中监测"地质云"的"全云化"监控的新模式,开创了一种云端资源监控的新思路;基于大数据技术设计了一个透明化的地质云监控平台,采用Hadoop、HDFS、HBase等研发了一种生于"地质云"中、存在于"地质云"内、全部使命为监测"地质云"的崭新的监控软件与分析系统,可完成地质云监控数据的一站式采集、解析、存储、展示、查询、分析、挖掘与服务.经在线运行结果表明,地质云监控平台将监控工作量缩减为原来的四分之一,能够自动地完成业务服务情况分析、稳定性评价、故障和错误诊断、问题点快速定位等功能,为"地质云"稳定运行提供了重要保障.

Geo Cloud 1. 0 is first comprehensive integrated application platform in geological survey field in China. It was released publicly and begun to run online since November 2017,and it can offer onestop service for geosciences data integration,geological business integration and product application integration. In order to ensure Geo Cloud run smoothly and provide high quality services for end user continuously,it is necessary to monitor about 400 servers with more than ten thousand indexes real-timely. This paper analyzes the interrelation of hardware and software resources of Geo Cloud deeply,determines the quantitative monitoring indexes of all types of resources in this national geological cloud,and proposes a new innovative all-in-cloud work pattern for monitoring the geological cloud computing first time. The new idea opens a new kind of method to monitor resources in the cloud. A transparent Geo Cloud Monitoring Platform（ GMP） is designed based on big data technology,our team developed the corresponding monitor big data algorithms and applications using Hadoop,HDFS,HBase. The platform and application are born in the cloud,exist in the cloud and they aim to supervise various servers. The one-stop software can complete the whole lifecycle of monitoring data includes acquisition,recognition,storage,display,query,analysis,mining and services. The online operation results show that GMP can reduce the workload of administrative staff of Geo Cloud vastly. And even one-fourth people can perform original workload, such as analyzing the effect of geological services, evaluating the stability of geological cloud computing system rapidly,diagnosing fault or error of Geo Cloud intelligently,and locating the positions of problems accurately. In short,GMP plays a very significant role on operating smoothly for Geo Cloud.