摘要
为解决气象观察数据的频繁写入和大数据存储的问题,设计私有云存储模型。该模型分为用户访问接口层、元数据存储层、实体数据存储层和关系数据库4层结构。采用HDFS集群分布式文件系统,实现气象大数据的存储和高效查询与搜索业务;使用关系数据库系统,实现数据采集端频繁数据写入的业务。实验结果表明,该平台实现了海量气象数据存储、查询统计和属性管理,克服了云存储单条插入效率低的缺点,较好满足了气象部门大数据存储和管理的需求。
To solve the problem of frequent meteorological observation data-write-in and data storage,a private cloud storage model was designed consisting of four layers：user access interface layer,meta-data storage layer,entity data storage layer and a relational database layer.Hadoop distributed file system was adopted to implement the meteorological data storage as well as efficient query and search business.Relational database was used to realize frequent data writing of data collection extremity.Results from experiments illustrate that this system not only achieves massive meteorological data storage,query and attribute management,but overcomes the shortcomings of single insertion inefficiency.It meets the requirements of the data storage and management of meteorological departments.
出处
《计算机工程与设计》
北大核心
2015年第2期426-430,435,共6页
Computer Engineering and Design
基金
内蒙古自然科学基金项目(2010MS0913)
内蒙古自治区高等学校科学研究基金项目(NJZY12052)
内蒙古工业大学重点基金项目(ZD201118)
关键词
大数据
云存储
分布式文件系统
关系数据库
列数据库
big data
cloud storage
distributed file system
relational database
column oriented database
