基于分布式技术的气象大数据共享服务系统设计与实现

DESIGN AND REALISATION METEOROLOGICAL BIG DATA SHARING SERVICE SYSTEM BASED ON DISTRIBUTED TECHNOLOGY

随着气象数据的不断增长及气象服务现代化要求的提升,原有省级CIMISS系统已经无法满足业务需求。为了解决系统动态扩展能力不足、实时查询性能低下等问题,分别利用分布式网格存储系统和分布式数据库,以及分布式文件系统和NAS技术,构建基于分布式技术的气象大数据共享服务系统,实现结构化数据和半/非结构化数据的存储、共享和管理,并实现与CIMISS系统无缝对接。实践证明,该方案能极大提升CIMISS系统的数据存储能力,且并发响应能力强。相同条件下,查询性能提升22倍以上,能够有效满足未来气象业务对数据服务的要求。

With the continuous growth of meteorological data and the improvement of meteorological service modernization requirements, the original provincial CIMISS system is unable to meet the business needs. In order to solve the problems of insufficient dynamic expansion ability and low real-time query performance of the system, this paper used the distributed grid storage system, the distributed database, the distributed file system and NAS technology respectively to build the meteorological big data sharing service system based on the distributed technology. The system realized the storage, sharing and management of structured data and semi/unstructured data, and realized the seamless connection with CIMISS system. It has been proved that this scheme can greatly improve the data storage ability of CIMISS system and has strong concurrent response ability. Under the same conditions, the query performance is improved by more than 22 times, which can effectively meet the requirements of future meteorological services for data services.