基于大数据技术的质控数据服务系统实现

Implementation of Quality Control Data Service System Based on Big Data Technology

现有气象系统的设计和建设缺乏统一的规划,导致各系统功能全、规模小,整体的安全防护能力较弱,集约化差,“信息孤岛”现象仍然存在。论文利用大数据技术,搭建了以MySQL、GBase、HBase和分布式NAS为基础的统一数据存储环境,建设了融入业务体系的“一中心、一平台、多应用”的天津气象大数据共享网,形成了支撑天津省局特色的全业务数据服务中心,有效实现了数据的共享,流程的高度集约,极大提升了省级气象综合业务能力和数据的服务能力。实际测试表明,GBase数据库查询性能提升5倍多,而随着时间延长,甚至提升至22倍以上,HBase的查询性能在毫秒级,MySQL的查询性能在秒级。

The lack of a unified planning for the design and construction of the existing meteorological system has resulted in the fact that the functions of each system are complete,the scale is small,the overall safety protection capability is weak,the intensification is poor,and the phenomenon of"information island"still exists.In this paper,the big data technology is used to build a unified data storage environment based on MySQL,GBase,HBase and distributed NAS,and build a unified data storage environment integrated into the business system Tianjin Meteorological big data sharing network of"one center,one platform and multiple applications"has formed a full service data service center supporting the characteristics of Tianjin Provincial Bureau,effectively realizing data sharing,highly intensive process,and greatly improving the provincial meteorological comprehensive business ability and data service ability.The actual test shows that the query performance of gbase database has been improved by more than 5 times,but with the time extension,even more than 22 times,the query performance of HBase is in milliseconds,and that of MySQL is in seconds.