云数据存储技术在气象数据存储中的应用

APPLICATION OF CLOUD DATA STORAGE TECHNOLOGY IN METEOROLOGICAL DATA STORAGE

气象数据具有体量巨大、种类繁多、数据响应速度要求快等特点,传统的数据存储模式难以适应气象大数据的需求。基于云分布式存储技术设计一套数据存储方法,提高了气象数据存储传输能力。将气象数据分成结构化、半结构化和非结构化三种结构类型,针对不同结构类型的数据分别选择云存储模型,结构化数据采用分布式关系型数据库存储,半/非结构化数据采用No SQL数据库存储。在对半/非结构化数据存储设计中,对数据进行主键表及属性表设计,对属性表的数据进行分块和压缩,提高大数据的存储读取性能。根据数据的时间度和使用度原则阐述了不同类型气象数据间的转换。实验表明,设计的基于云分布式存储技术的数据存储方法是可行的,在一定程度上提高了气象数据存储传输能力,为气象数据的云化建设提供一定的借鉴和参考。

The meteorological data have the characteristics of huge volume, various types and fast response to the data. The traditional data storage mode is difficult to meet the demand of large meteorological data. A set of data storage methods was designed based on cloud distributed storage technology to improve the storage and transmission capacity of meteorological data.The meteorological data were divided into three structural types： structured, semi-structured and unstructured. Different structural types of data chose different cloud storage models. Structured data were stored in distributed relational database. Semi-structured data and unstructured data were stored in NoSQL database. In the design of Semi-structured data and unstructured data storage, we designed the primary key table and the attribute table, blocked and compressed data of the attribute table to improve the storage and reading performance of large data.Moreover,the conversion of different types of meteorological data was explained according to the time and used the degree of the data. Experiments show that the data storage method based on cloud distributed storage technology is feasible. To a certain extent, it improves the transmission capacity of meteorological data storage.The data storage method discussed in the paper provides a reference for the cloud construction of meteorological data.